Abstract
Detecting and tracking people is becoming more important in robotic applications because of the increasing demand for collaborative work in which people interact closely with and in the same workspace as robots. New safety standards allow people to work next to robots, but require that they be protected from harm while they do so. Sensors that detect and track people are a natural way of implementing the necessary safety monitoring, and have the added advantage that the information about where the people are and where they are going can be fed back into the application and used to give the robot greater situational awareness for performing tasks. The results should help users determine if such a system will provide sufficient protection for people to be able to work safely in collaborative applications with industrial robots.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
ISO TC 184: ISO 10218-2:2011 Robots and robotic devices – Safety requirements for industrial robots – Part 2: Robot systems and integration. In. (2011)
ISO TC 184: ISO 10218-1:2011 Robots and robotic devices – Safety requirements for industrial robots – Part 1: Robots. In. (2011)
American National Standards Institute, Robotics Industries Association: ANSI/RIA R15.06-2012, Industrial Robots and Robot Systems - Safety Requirements. In. (2012)
Shneier, M., Hong, T., Cheok, G., Saidi, K., Shackleford, W.: Performance evaluation methods for human detection and tracking systems for robotic applications. In., vol. NISTIR 8045. National Institute of Standards and Technology, Gaithersburg (2015)
Ogale, N.A.: A survey of techniques for human detection from video. University of Maryland (2006)
Ferryman, J., Crowley, J.L.: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance. In: Ferryman, J., Crowley, J.L. (eds.) (2013)
Nascimento, J.C., Marques, J.S.: Performance evaluation of object detection algorithms for video surveillance. IEEE Trans. Multimedia 8(4), 761–774 (2006). doi:10.1109/TMM.2006.876287
Stiefelhagen, R., Garofolo, J.: Multimodal Technologies for Perception of Humans, First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006, Lecture Notes in Computer Science, vol. 4122. Springer, Southampton (2007)
Nghiem, A.T., Bremond, F., Thonnat, M., Valentin, V.: ETISEO, performance evaluation for video surveillance systems. In: IEEE Conference on Advanced Video and Signal Based Surveillance, 2007. AVSS 2007, pp. 476–481 (2007)
Home Office Centre for Applied Science and Technology: Imagery Library for Intelligent Detection Systems (i-LIDS): The i-LIDS User Guide. In. (2011)
British Home Office: Imagery Library for Intelligent Detection Systems. https://www.gov.uk/imagery-library-for-intelligent-detection-systems (2013). Accessed 30 May 2014
Brown, L.M., Senior, A.W., Tian, Y., Connell, J., Hampapur, A., Shu, C., Merkl, H., Lu, M.: Performance evaluation of surveillance systems under varying conditions. Paper presented at the IEEE international workshop on performance evaluation of tracking and surveillance, Colorado
Yin, F., Makris, D., Velastin, S.: Performance evaluation of object tracking algorithms. Paper presented at the 10th IEEE international workshop on performance evaluation of tracking and surveillance (PETS 2007), Rio de Janeiro
Lazarevic-McManus, N., Renno, J.R., Makris, D., Jones, G.A.: An object-based comparative methodology for motion detection based on the F-Measure. Comput. Vis. Image Underst. 111(1), 74–85 (2008). doi:10.1016/j.cviu.2007.07.007
Bashir, F., Porikli, F.: Performance evaluation of object detection and tracking systems. Paper presented at the 9th IEEE international workshop on performance evaluation of tracking and surveillance (PETS 2006), New York
Black, J., Ellis, T., Rosin, P.: A novel method for video tracking performance evaluation. Paper presented at the joint IEEE international workshop on visual surveillance and performance evaluation of tracking and surveillance, Nice, 2003
Davis, J., Goadrich, M.: The relationship between Precision-Recall and ROC curves. Paper presented at the Proceedings of the 23rd international conference on machine learning, Pittsburgh, 2006
Kalal, Z., Matas, J., Mikolajczyk, K.: Online learning of robust object detectors during unstable tracking. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1417–1424 (2009)
Popoola, J., Amer, A.: Performance evaluation for tracking algorithms using object labels. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, pp. 733–736 (2008)
Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment. Paper presented at the The 6th IEEE international workshop on visual surveillance, VS 2006, Graz
Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2), 90–126 (2006). doi:10.1016/j.cviu.2006.08.002
Baumann, A., Bolz, M., Ebling, J., Koenig, M., Loos, H.S., Merkel, M., Niem, W., KarlWarzelhan, J., Yu, J.: A review and comparison of measures for automatic video surveillance systems. EURASIP Journal on Image and Video Processing 2008 (2008). doi:10.1155/2008/824726
Stiefelhagen, R., Bowers, R., Fiscus, J.G.: Multimodal Technologies for Perception of Humans, International Evaluation Workshops CLEAR 2007 and RT 2007, Baltimore, MD, USA, May 8–11, 2007, Revised Selected Papers. Lecture Notes in Computer Science. Springer, Berlin Heidelberg (2008)
Bodt, B., Camden, R., Scott, H., Jacoff, A., Hong, T., Chang, T., Norcross, R., Downs, T., Virts, A.: Performance measurements for evaluating static and dynamic multiple human detection and tracking systems in unstructured environments. Paper presented at the Proceedings of the 9th workshop on performance metrics for intelligent systems, Gaithersburg, Maryland, 2009
Godil, A., Bostelman, R., Saidi, K., Shackleford, W., Cheok, G., Shneier, M., Hong, T.: 3D ground-truth systems for object/human recognition and tracking. In: 2013 IEEE Conference on 2013 Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 719–726. IEEE
Nikon Corporation: iGPS/iSpace. http://www.nikon.com/products/instruments/lineup/industrial/metrology_assisted_production/large_volume/igps/index.htm. Accessed 2 June 2014
Schmitt, R., Nisch, S., Schönberg, A., Demeester, F., Renders, S.: Performance evaluation of iGPS for industrial applications. In: 2010 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1–8 (2010)
Mosqueira, G., Apetz, J., Santos, K.M., Villani, E., Suterio, R., Trabasso, L.G.: Analysis of the indoor GPS system as feedback for the robotic alignment of fuselages using laser radar measurements as comparison. Robot. Comput.-Integr. Manuf. 28(6), 700–709 (2012). doi:10.1016/j.rcim.2012.03.004
Wang, Z., Mastrogiacomo, L., Franceschini, F., Maropoulos, P.G.: Experimental comparison of dynamic tracking performance of iGPS and laser tracker. Int. J. Adv. Manuf. Technol. 56(1–4), 205–213 (2011). doi:10.1007/s00170-011-3166-0
Depenthal, C.: Path tracking with IGPS. In: International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1–6 (2010)
Chambers, D.R., Flannigan, C., Wheeler, B.: High-accuracy real-time pedestrian detection system using 2D and 3D features. SPIE Proceedings Three-Dimensional Imaging, Visualization, and Display, vol. 83840G, pp. 83840G-83841–83840G-83811 (2012)
Dalal, N.: The INRIA Person Dataset. http://pascal.inrialpes.fr/data/human/ (2005)
Papageorgiou, C.P., Oren, M., Poggio, T.: A general framework for object detection. In: 6th International Conference on Computer Vision, pp. 555–562 (1998)
Freund, Y., Schapire, R.E.: A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5), 771–780 (1999)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995). doi:10.1007/BF00994018
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, pp. 886–893 (2005)
Shackleford, W.: Position tracking performance metrics. https://github.com/usnistgov/PTPM (2014)
ISO TC 199: ISO 13855:2010 Safety of machinery – Positioning of safeguards with respect to the approach speeds of parts of the human body. In: International Organization for Standardization (ed.) (2010)
Bodt, B., Hong, T.: UGV safe operations capstone experiment. Paper presented at the Army science conference, Orlando, 2010
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shackleford, W., Cheok, G., Hong, T. et al. Performance Evaluation of Human Detection Systems for Robot Safety. J Intell Robot Syst 83, 85–103 (2016). https://doi.org/10.1007/s10846-016-0334-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-016-0334-3