Abstract
In this paper, a pipeline for hand gesture recognition from depth images is presented. This depth-based image recognition system is capable of recognizing gestures with challenges like varying depths, complex backgrounds, and variation in view point, hand pose, and appearance. Firstly, we obtain a grayscale image from the depth map, segment the hand region, and perform orientation normalization and feature extraction, which is followed by classification. Two different sets of feature descriptors are extracted: Multi-Radii Circular Signatures (MRCS) and Multi-Scale Density (MSD). Different classifiers have been used to demonstrate the efficacy of the suggested pipeline. Overall accuracy of 98.90% (MRCS) and 99.78% (MSD) is obtained using the MLP classifier.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Suarez J, Murphy RR (2012) Hand gesture recognition with depth images: a review. In: 2012 IEEE RO-MAN: the 21st IEEE international symposium on robot and human interactive communication. IEEE, pp 411–417
Garg P, Aggarwal N, Sofat S (2009) Vision based hand gesture recognition. Int J Comput Inf Eng 3(1):186–191
Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from single depth images. In: CVPR 2011. IEEE, pp 1297–1304
She Y, Wang Q, Jia Y, Gu T, He Q, Yang B (2014) A real-time hand gesture recognition approach based on motion features of feature points. In: 2014 IEEE 17th international conference on computational science and engineering. IEEE, pp 1096–1102
Kapuscinski T, Oszust M, Wysocki M (2013) Recognition of signed dynamic expressions observed by tof camera. In: 2013 signal processing: algorithms, architectures, arrangements, and applications (SPA). IEEE, pp 291–296
Ali HH, Moftah HM, Youssif AA (2018) Depth-based human activity recognition: a comparative perspective study on feature extraction. Future Comput Inf J 3(1):51–67
Bakheet S, Al-Hamadi A (2021) Robust hand gesture recognition using multiple shape-oriented visual cues. EURASIP J Image Video Process 2021(1):1–18
Cao J, Yu S, Liu H, Li P (2016) Hand posture recognition based on heterogeneous features fusion of multiple kernels learning. Multimedia Tools Appl 75:11909–11928
Triesch J, Von Der Malsburg C (2001) A system for person-independent hand posture recognition against complex backgrounds. IEEE Trans Pattern Anal Mach Intell 23(12):1449–1453
Pisharady PK, Vadakkepat P, Loh AP (2013) Attention based detection and recognition of hand postures against complex backgrounds. Int J Comput Vis 101:403–419
Ren Y, Xie X, Li G, Wang Z (2016) Hand gesture recognition with multiscale weighted histogram of contour direction normalization for wearable applications. IEEE Trans Circ Syst Video Technol 28(2):364–377
Zhan F (2019) Hand gesture recognition with convolution neural networks. In: 2019 IEEE 20th international conference on information reuse and integration for data science (IRI). IEEE, pp 295–298
Adithya V, Rajesh R (2020) A deep convolutional neural network approach for static hand gesture recognition. Procedia Comput Sci 171:2353–2361
Stergiopoulou E, Sgouropoulos K, Nikolaou N, Papamarkos N, Mitianoudis N (2014) Real time hand detection in a complex background. Engi Appl Artif Intell 35:54–70
Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43:1–54
Muhammad H, Saud A, Shafiq A, Mazen Z, Shamsul H, Sofia I (2022) Hand gesture recognition with symmetric pattern under diverse illuminated conditions using artificial neural network. Symmetry 14(10):2045
Dominio F, Donadeo M, Marin G, Zanuttigh P, Cortelazzo GM (2013) Hand gesture recognition with depth data. In: Proceedings of the 4th ACM/IEEE international workshop on analysis and retrieval of tracked events and motion in imagery stream, pp 9–16
Wan T, Wang Y, Li J (2012) Hand gesture recognition system using depth data. In: 2012 2nd international conference on consumer electronics, communications and networks (CECNet). IEEE, pp 1063–1066
Wang C, Liu Z, Chan SC (2014) Superpixel-based hand gesture recognition with kinect depth camera. IEEE Trans Multimedia 17(1):29–39
Yao Y, Fu Y (2014) Contour model-based hand-gesture recognition using the kinect sensor. IEEE Trans Circ Syst Video Technol 24(11):1935–1944
Yang L, Longyu L, Dong H, Alelaiwi A, El Saddik A (2015) Evaluating and improving the depth accuracy of kinect for windows v2. IEEE Sens J 15(8):4275–4285
Gaber A, Faher MF, Waned MA (2015) Automated grading of facial paralysis using the kinect v2: a proof of concept study. In: 2015 international conference on virtual rehabilitation (ICVR). IEEE, pp 258–264
Abdulateef SK, Salman MD (2021) A comprehensive review of image segmentation techniques. Iraqi J Electr Electr Eng 17(2)
Liao PS, Chen TS, Chung PC et al (2001) A fast algorithm for multilevel thresholding. J Inf Sci Eng 17(5):713–727
Huang DY, Lin TW, Hu WC (2011) Automatic multilevel thresholding based on two-stage otsu’s method with cluster determination by valley estimation. Int J Innov Comput Inf Control 7(10):5631–5644
Sahana T, Basu S, Nasipuri M, Mollah AF (2022) Mrcs: multi-radii circular signature based feature descriptor for hand gesture recognition. Multimedia Tools Appl 81(6):8539–8560
Sahana T, Paul S, Basu S, Mollah AF (2020) Hand sign recognition from depth images with multi-scale density features for deaf mute persons. Procedia Comput Sci 167:2043–2050
Memo A, Minto L, Zanuttigh P (2015) Exploiting silhouette descriptors and synthetic data for hand gesture recognition. Smart Tools Apps Graph 15–23
Memo A, Zanuttigh P (2018) Head-mounted gesture controlled interface for human-computer interaction. Multimedia Tools Appl 77(1):27–53
Tang H, Wang W, Xu D, Yan Y, Sebe N (2018) Gesturegan for hand gesture-to-gesture translation in the wild. In: Proceedings of the 26th ACM international conference on multimedia, pp 774–782
Liu Y, De Nadai M, Zen G, Sebe N, Lepri B (2019) Gesture-to-gesture translation in the wild via category-independent conditional maps. In: Proceedings of the 27th ACM international conference on multimedia, pp 1916–1924
Miah AS, Hasan MA, Shin J, Okuyama Y, Tomioka Y (2023) Multistage spatial attention-based neural network for hand gesture recognition. Computers 12(1):13
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sahana, T., Mollah, A.F. (2023). An Effective Pipeline for Depth Image-Based Hand Gesture Recognition. In: Das, A.K., Nayak, J., Naik, B., Vimal, S., Pelusi, D. (eds) Computational Intelligence in Pattern Recognition. CIPR 2022. Lecture Notes in Networks and Systems, vol 725. Springer, Singapore. https://doi.org/10.1007/978-981-99-3734-9_40
Download citation
DOI: https://doi.org/10.1007/978-981-99-3734-9_40
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-3733-2
Online ISBN: 978-981-99-3734-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)