Abstract
Assistive robot manipulators must be able to autonomously pick and place a wide range of novel objects to be truly useful. However, current assistive robots lack this capability. Additionally, assistive systems need to have an interface that is easy to learn, to use, and to understand. This paper takes a step forward in this direction. We present a robot system comprised of a robotic arm and a mobility scooter that provides both pick-and-drop and pick-and-place functionality for open world environments without modeling the objects or environment. The system uses a laser pointer to directly select an object in the world, with feedback to the user via projecting an interface into the world. Our evaluation over several experimental scenarios shows a significant improvement in both runtime and grasp success rate relative to a baseline from the literature [5], and furthermore demonstrates accurate pick and place capabilities for tabletop scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Achic, F., Montero, J., Penaloza, C., Cuellar, F.: Hybrid BCI system to operate an electric wheelchair and a robotic arm for navigation and manipulation tasks. In: 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), pp. 249–254, July 2016
Diankov, R., Kuffner, J.: OpenRAVE: a planning architecture for autonomous robotics. Robotics Institute, Pittsburgh, PA, CMU-RI-TR-08-34, 79 (2008)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Grice, P.M., Kemp, C.C.: Assistive mobile manipulation: designing for operators with motor impairments. In: RSS 2016 Workshop on Socially and Physically Assistive Robotics for Humanity (2016)
Gualtieri, M., Kuczynski, J., Shultz, A.M., Pas, A.T., Platt, R., Yanco, H.: Open world assistive grasping using laser selection. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 4052–4057, May 2017
Gualtieri, M., ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 598–605, October 2016
Hawkins, K.P.: Analytic inverse kinematics for the universal robots UR-5/UR-10 arms. Technical report, Georgia Institute of Technology (2013)
He, W., Goodkind, D., Kowal, P.: An aging world: 2015. International Population Reports (2016)
Jain, A., Kemp, C.C.: EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces. Auton. Robot. 28(1), 45 (2010)
Kalashnikov, D., et al.: QT-OPT: scalable deep reinforcement learning for vision-based robotic manipulation. arXiv preprint arXiv:1806.10293 (2018)
Kemp, C.C., Anderson, C.D., Nguyen, H., Trevor, A.J., Xu, Z.: A point-and-click interface for the real world: laser designation of objects for mobile manipulation. In: 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 241–248, March 2008
Kraus, L., Lauer, E., Coleman, R., Houtenville, A.: Disability statistics annual report. University of New Hampshire, Durham, NH, USA (2018)
Mahler, J., et al.: DEX-Net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:1703.09312 (2017)
Martens, C., Ruchel, N., Lang, O., Ivlev, O., Graser, A.: A friend for assisting handicapped people. IEEE Robot. Autom. Mag. 8(1), 57–65 (2001)
Morrison, D., Corke, P., Leitner, J.: Closing the loop for robotic grasping: a real-time, generative grasp synthesis approach. arXiv preprint arXiv:1804.05172 (2018)
Pathirage, I., Khokar, K., Klay, E., Alqasemi, R., Dubey, R.V.: A vision based P300 brain computer interface for grasping using a wheelchair-mounted robotic arm. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 188–193 (2013)
Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, Kobe, Japan, vol. 3, p. 5 (2009)
Rokach, L., Maimon, O.: Clustering methods. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, pp. 321–352. Springer, Boston (2005). https://doi.org/10.1007/b107408
Rusu, R.B.: Semantic 3D object maps for everyday manipulation in human living environments. Ph.D. thesis, Computer Science department, Technische Universitaet Muenchen, Germany, October 2009
Schulman, J., Ho, J., Lee, A.X., Awwal, I., Bradlow, H., Abbeel, P.: Finding locally optimal, collision-free trajectories with sequential convex optimization. In: Robotics: Science and Systems, vol. 9, pp. 1–10. Citeseer (2013)
ten Pas, A., Gualtieri, M., Saenko, K., Platt, R.: Grasp pose detection in point clouds. Int. J. Robot. Res. 36(13–14), 1455–1473 (2017)
Viereck, U., Pas, A., Saenko, K., Platt, R.: Learning a visuomotor controller for real world robotic grasping using simulated depth images. In: Conference on Robot Learning (CoRL) (2017)
Acknowledgements
This work has been supported in part by the National Science Foundation (IIS-1426968, IIS-1427081, IIS-1724191, IIS-1724257, IIS-1763469), NASA (NNX16AC48A, NNX13AQ85G), ONR (N000141410047), Amazon through an ARA to Platt, and Google through a FRA to Platt.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, D. et al. (2022). Towards Assistive Robotic Pick and Place in Open World Environments. In: Asfour, T., Yoshida, E., Park, J., Christensen, H., Khatib, O. (eds) Robotics Research. ISRR 2019. Springer Proceedings in Advanced Robotics, vol 20. Springer, Cham. https://doi.org/10.1007/978-3-030-95459-8_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-95459-8_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95458-1
Online ISBN: 978-3-030-95459-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)