Abstract
Simulation has been crucial for robotics research development almost from the beginning of its existence. While simulation has been widely used for education, testing, and prototyping, only very recently the robotics community has attempted transferring behaviors learned in simulation to the real world (this process is usually referred to as Sim2Real). Those attempts have opened-up a novel research direction that has produced some exciting results that were previously thought impossible to achieve. In this paper, we attempt to give a quick overview of the most promising Simulation-To-Reality (Sim2Real) methods, results and directions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ajay, A., et al.: Combining physical simulators and object-based networks for control. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 3217–3223. IEEE (2019)
Akkaya, I., et al.: Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113 (2019)
Alghonaim, R., Johns, E.: Benchmarking domain randomisation for visual sim-to-real transfer. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 12802–12808. IEEE (2021)
Battaglia, P.W., Pascanu, R., Lai, M., Rezende, D., Kavukcuoglu, K.: Interaction networks for learning about objects, relations and physics. In: Conference on Neural Information Processing Systems (NeurIPS) (2016)
Bousmalis, K., et al.: Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 4243–4250. IEEE (2018)
Chatzilygeroudis, K., Mouret, J.B.: Using parameterized black-box priors to scale up model-based policy search for robotics. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 5121–5128. IEEE (2018)
Chatzilygeroudis, K., Rama, R., Kaushik, R., Goepp, D., Vassiliades, V., Mouret, J.B.: Black-box data-efficient policy search for robotics. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 51–58. IEEE (2017)
Chatzilygeroudis, K., Vassiliades, V., Mouret, J.B.: Reset-free trial-and-error learning for robot damage recovery. Robot. Autonom. Syst. 100, 236–250 (2018)
Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., Mouret, J.B.: A survey on policy search algorithms for learning robot controllers in a handful of trials. IEEE Trans. Robot. 36(2), 328–347 (2019)
Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., Fox, D.: Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 8973–8979. IEEE (2019)
Gangapurwala, S., Geisert, M., Orsolino, R., Fallon, M., Havoutis, I.: Rloc: Terrain-aware legged locomotion using reinforcement learning and optimal control. arXiv preprint arXiv:2012.03094 (2020)
Gupta, A., Kumar, V., Lynch, C., Levine, S., Hausman, K.: Relay policy learning: solving long horizon tasks via imitation and reinforcement learning. In: Conference on Robot Learning (CoRL) (2019)
Heess, N., et al.: Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017)
Ho, J., Ermon, S.: Generative adversarial imitation learning. Conf. Neural Inf. Process. Syst. (NeurIPS) 29, 4565–4573 (2016)
Isermann, R.: Fault-Diagnosis Systems: An Introduction From Fault Detection to Fault tolerance. Springer Science & Business Media, Cham (2005)
James, S., Davison, A.J., Johns, E.: Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task. In: Conference on Robot Learning, pp. 334–343. PMLR (2017)
James, S., et al.: Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12627–12637 (2019)
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., Hutter, M.: Learning quadrupedal locomotion over challenging terrain. Sci. Robot. 5(47), eabc5986 (2020)
Lee, T.E., et al.: Camera-to-robot pose estimation from a single image. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 9426–9432. IEEE (2020)
Mahler, J., et al.: Learning ambidextrous robot grasping policies. Sci. Robot.4(26), eaau4984 (2019)
Mouret, J.B., Chatzilygeroudis, K.: 20 years of reality gap: a few thoughts about simulators in evolutionary robotics. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1121–1124 (2017)
Muratore, F., Treede, F., Gienger, M., Peters, J.: Domain randomization for simulation-based policy optimization with transferability assessment. In: Conference on Robot Learning, pp. 700–713. PMLR (2018)
Ota, K., Jha, D.K., Romeres, D., van Baar, J., Smith, K.A., Semitsu, T., Oiki, T., Sullivan, A., Nikovski, D., Tenenbaum, J.B.: Data-efficient learning for complex and real-time physical problem solving using augmented simulation. IEEE Robot. Autom. Lett. 6(2), 4241–4248 (2021)
Rajeswaran, A., Ghotra, S., Ravindran, B., Levine, S.: Epopt: Learning robust neural network policies using model ensembles. In: International Conference on Learning Representations (ICLR) (2017)
Siciliano, B., Khatib, O.: Springer Handbook of Robotics. Springer, Berlin (2016)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, London (2018)
Tanwani, A.K.: Domain invariant representation learning for sim-to-real transfer. In: Conference on Robot Learning (2020)
Zhang, F., Leitner, J., Ge, Z., Milford, M., Corke, P.: Adversarial discriminative sim-to-real transfer of visuo-motor policies. Int. J. Robot. Res. 38(10–11), 1229–1245 (2019)
Zhao, W., Queralta, J.P., Westerlund, T.: Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 737–744. IEEE (2020)
Zhu, Y., et al.: Reinforcement and imitation learning for diverse visuomotor skills. In: Proceedings of Robotics: Science and Systems. Pittsburgh, Pennsylvania, June 2018. https://doi.org/10.15607/RSS.2018.XIV.009
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dimitropoulos, K., Hatzilygeroudis, I., Chatzilygeroudis, K. (2022). A Brief Survey of Sim2Real Methods for Robot Learning. In: Müller, A., Brandstötter, M. (eds) Advances in Service and Industrial Robotics. RAAD 2022. Mechanisms and Machine Science, vol 120. Springer, Cham. https://doi.org/10.1007/978-3-031-04870-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-04870-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04869-2
Online ISBN: 978-3-031-04870-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)