Abstract
Despite a lot of research in the field, only very little experience exists with Teaching by Demonstration (TbD) in actual industrial use cases. In the factory of the future, it is necessary to rapidly reprogram flexible mobile manipulators to perform new tasks, when the need arises, for which a working system capable of TbD would be ideal. Contrary to current TbD approaches, that generally aim to recognize both action and where it is applied, we propose a division of labor, where the operator manually specifies the action the robot should perform, while gestures are used for specifying the relevant action parameter (e.g. on which object to apply the action). Using this two-step method has the advantages that there is no uncertainty of which action the robot will perform, it takes into account that the environment changes, so objects do not need to be at predefined locations, and the parameter specification is possible even for inexperienced users. Experiments with 24 people in 3 different environments verify that it is indeed intuitive, even for a robotics novice, to program a mobile manipulator using this method.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Factory-in-a-Day project (2014). http://www.factory-in-a-day.seu/
STAMINA project (2014). http://stamina-robot.eu
TAPAS project (2014). http://tapas-project.eu
Archibald, C.C.: A computational model for skills-oriented robot programming. PhD thesis, University of Ottawa, Ottawa (1995)
Bøgh, S., Nielsen, O.S., Pedersen, M.R., Krüger, V., Madsen, O.: Does your robot have skills? In: Proceedings of the 43rd International Symposium on Robotics (ISR), Taipei (2012)
Bischoff, R., Kazi, A.: Perspectives on augmented reality based human-robot interaction with industrial robots. In: Intelligent Robots and Systems, 2004.(IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on, vol. 4, pp. 3226–3231 (2004)
Björkelund, A., Edstrom, L., Haage, M., Malec, J., Nilsson, K., Nugues, P., Robertz, S., Storkle, D., Blomdell, A., Johansson, R., Linderoth, M., Nilsson, A., Robertsson, A., Stolt, A., Bruyninckx, H.: On the integration of skilled robot motions for productivity in manufacturing. In: 2011 IEEE International Symposium on Assembly and Manufacturing (ISAM). doi:10.1109/ISAM.2011.5942366 (2011)
Bobick, A.: On human action. In: Moeslund, T.B., Hilton, A., Krüger, V., Sigal, L. (eds.) Visual analysis of humans, pp. 279–288. Springer, London (2011)
Bobick, A.F.: Movement, activity and action: the role of knowledge in the perception of motion. Phil. Trans. R. Soc. B Biol. Sci. 352(1358), 1257–1265 (1997). PMID: 9304692 PMCID: 1692010
Bruyninckx, H., De Schutter, J.: Specification of force-controlled actions in the “task frame formalism”-a synthesis. IEEE Trans. Robot. Autom. 12(4), 581–589 (1996). doi:10.1109/70.508440
Chen, H., Sheng, W.: Transformative CAD based industrial robot program generation. Robot. Comput. Integr. Manuf. 27(5), 942–948 (2011). doi:10.1016/j.rcim.2011.03.006
Ekvall, S., Kragic, D.: Robot learning from demonstration: a task-level planning approach. Int. J. Adv. Robot. Syst. 5(3), 223–234 (2008)
Finkemeyer, B., Kröger, T., Wahl, F.M.: Executing assembly tasks specified by manipulation primitive nets. Adv. Robot. 19(5), 591–611 (2005)
Gadensgaard, D., Bourne, D.: Human/Robot multi-initiative setups for assembly cells. In: ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems, pp. 1–6 (2011)
Galindo, C., Fernández-Madrigal, J.A., Gonzílez, J., Saffiotti, A.: Robot task planning using semantic maps. Robot. Auton. Syst. 56(11), 955–966 (2008). doi:10.1016/j.robot.2008.08.007
Gottschlich, S., Ramos, C., Lyons, D.: Assembly and task planning: a taxonomy. IEEE Robot. Autom. Mag. 1(3), 4–12 (1994)
Hartmann, M.: DYNAPRO: Erfolgreich produzieren in turbulenten Märkten. Logis Verlag (1996)
Hartmann, M.: DYNAPRO II: erfolgreich produzieren in turbulenten Märkten. Logis Verlag (1997)
Hartmann, M.: DYNAPRO III: Erfolgreich produzieren in turbulenten Märkten. Logis Verlag (1998)
Høilund, C., Krüger, V., Moeslund, T.: Evaluation of human body tracking system for gesture-based programming of industrial robots. In: Proceedings of the 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 477–480. IEEE (2012)
Huckaby, J., Vassos, S., Christensen, H.I.: Planning with a task modeling framework in manufacturing robotics. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5787–5794 (2013)
Hvilshøj, M., Bøgh, S., Nielsen, O.S., Madsen, O.: Autonomous industrial mobile manipulation (AIMM): past, present and future. Ind. Robot. Int. J. 39(2), 120–135 (2012). doi:10.1108/01439911211201582
Inamura, T., Toshima, I., Tanie, H.: Embodied symbol emergence based on mimesis theory, vol. 23, pp. 4–5 (2004). http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.100.7250
Jen, Y.H., Taha, Z., Vui, L.J.: VR-Based robot programming and simulation system for an industrial robot. Int. J. Ind. Eng. Theory Appl. Pract. 15(3), 314–322 (2008)
Kang, S.B., Ikeuchi, K.: Toward automatic robot instruction from perception-temporal segmentation of tasks from human hand motion. IEEE Trans. Robot. Autom. 11(5), 670–681 (1995). doi:10.1109/70.466599
Kang, S.B., Ikeuchi, K.: Toward automatic robot instruction from perception-mapping human grasps to manipulator grasps. IEEE Trans. Robot. Autom. 13(1), 81–95 (1997). doi:10.1109/70.554349
Krüger, N., Piater, J., Wörgötter, F., Geib, C., Petrick, R., Steedman, M., Ude, A., Asfour, T., Kraft, D., Omrcen, D., et al: A formal definition of object-action complexes and examples at different levels of the processing hierarchy (2009). Technical report. http://www.paco-plus.org
Kröger, T., Finkemeyer, B., Wahl, F.: Manipulation primitives — a universal interface between sensor-based motion control and robot programming. In: Schütz, D., Wahl, F. (eds.) Robotic systems for handling and assembly, Springer Tracts in Advanced Robotics, vol. 67, pp. 293–313. Springer, Berlin (2011)
Kröger, T., Finkemeyer, B., Winkelbach, S., Eble, L.O., Molkenstruck, S., Wahl, F.: A manipulator plays jenga. IEEE Robot. Autom. Mag. 15(3), 79–84 (2008). doi:10.1109/MRA.2008.921547
Krüger, V., Herzog, D., Baby, S., Ude, A., Kragic, D.: Learning actions from observations. IEEE Robot. Autom. Mag. 17(2), 30–43 (2010). doi:10.1109/MRA.2010.936961
Krüger, V., Kragic, D., Ude, A., Geib, C.: The meaning of action: a review on action recognition and mapping. Adv. Robot. 21(13), 1473–1501 (2007)
Kulic, D., Kragic, D.: Learning action primitives. In: Moeslund, T.B., Hilton, A., Krüger V., Sigal, L. (eds.) Visual analysis of humans, pp. 333–354. Springer, London (2011)
Lopes, M., Santos-Victor, J.: A developmental roadmap for learning by imitation in robots. IEEE Trans. Syst. Man Cybern. B (Cybernetics) 37, 308–321 (2007). doi:10.1109/TSMCB.2006.886949
Madsen, O., Bøgh, S., Schou, C., Andersen, R., Damgaard, J., Pedersen, M.R., Krüger, V.: Integration of two autonomous mobile manipulators in a real-world industrial setting. In: IROS Workshop on Robotic Assistance Technologies in Industrial Settings (RATIS). Tokyo (2013)
Mitsi, S., Bouzakis, K.D., Mansour, G., Sagris, D., Maliaris, G.: Off-line programming of an industrial robot for manufacturing. Int. J. Adv. Manuf. Technol. 26(3), 262–267 (2005). doi:10.1007/s00170-003-1728-5
Neto, P., Mendes, N.: Direct off-line robot programming via a common CAD package. Robot. Auton. Syst. 61(8), 896–910 (2013). doi:10.1016/j.robot.2013.02.005
Pedersen, M.R., Herzog, D., Krüger, V.: Intuitive skill-level programming of industrial handling tasks on a mobile manipulator. 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2014)
Pedersen, M.R., Nalpantidis, L., Bobick, A., Krüger, V.: On the integration of hardware-abstracted robot skills for use in industrial scenarios. In: 2nd International IROS workshop on cognitive robotics systems: replicating human actions and activities. Tokyo (2013). http://renaud-detry.net/events/crs2013/papers/Pedersen.pdf
Takashi, I.A.: Towards an assembly plan from observation: task recognition with polyhedral objects. IEEE Trans. Robot. Autom. 10(3), 368–385 (1994) http://citeseerx.ist.psu.edu/viewdoc/summary?doi= 10.1.1.12.8697
Tenorth, M., Perzylo, A., Lafrenz, R., Beetz, M.: Representation and exchange of knowledge about actions, objects, and environments in the RoboEarth framework. IEEE Trans. Autom. Sci. Eng. 10 (3), 643–651 (2013). doi:10.1109/TASE.2013.2244883
Waibel, M., Beetz, M., Civera, J., D’Andrea, R., Elfring, J., Galvez-Lopez, D., Haussermann, K., Janssen, R., Montiel, J.M.M., Perzylo, A., Schiessle, B., Tenorth, M., Zweigle, O., van de Molengraft, R.: RoboEarth. IEEE Robot. Autom. Mag. 18(2), 69–82 (2011). doi:10.1109/MRA.2011.941632
Yeasin, M., Chaudhuri, S.: Toward automatic robot programming: learning human skill from visual data. IEEE Trans. Syst. Man Cybern. B Cybern. 30(1), 180–185 (2000)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pedersen, M.R., Krüger, V. Gesture-Based Extraction of Robot Skill Parameters for Intuitive Robot Programming. J Intell Robot Syst 80 (Suppl 1), 149–163 (2015). https://doi.org/10.1007/s10846-015-0219-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-015-0219-x