Abstract
Vision based micro gesture recognition systems enable the development of HCI (Human Computer Interaction) interfaces to mirror real-world experiences. It is unlikely that a gesture recognition method will be suitable for every application, as each gesture recognition system rely on the user cultural background and application domain. This research is an attempt to develop a micro gesture recognition system suitable for the asian culture. However, hands vary in shapes and sizes while gesture varies in orientation and motion. For accurate feature extraction, deep learning approaches are considered. Here, an integrated CNN-LSTM (Convolutional Neural Network- Long Short-Term Memory) model is proposed for building micro gesture recognition system. To demonstrate the applicability of the system two micro hand gesture-based datasets namely, standard and local dataset consisting of ten significant classes are used. Besides, the model is tested against both augmented and unaugmented datasets. The accuracy achieved for standard data with augmentation is 99.0%, while the accuracy achieved for local data with augmentation is 96.1% by applying CNN-LSTM model. In case of both the datasets, the proposed CNN-LSTM model appears to perform better than the other pre-trained CNN models including ResNet, MobileNet, VGG16 and VGG9 as well as CNN excluding LSTM.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, T.U., Hossain, M.S., Alam, M.J., Andersson, K.: An integrated cnn-rnn framework to assess road crack. In: 22nd International Conference on Computer and Information Technology (ICCIT), pp. 1–6. IEEE (2019)
Ahmed, T.U., Hossain, S., Hossain, M.S., ul Islam, R., Andersson, K.: Facial expression recognition using convolutional neural network with data augmentation. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 336–341. IEEE (2019)
Akoum, A., Al Mawla, N., et al.: Hand gesture recognition approach for asl language using hand extraction algorithm. J. Softw. Eng. Appl. 8(08), 419 (2015)
Basnin, N., Hossain, M.S., Nahar, L.: An integrated cnn-lstm model for bangla lexical sign language recognition. In: Proceedings of 2nd International Conference on Trends in Computational and Cognitive Engineering (TCCE-2020) Springer Joint 8th International Conference on Informatics (2020)
Chowdhury, R.R., Hossain, M.S., ul Islam, R., Andersson, K., Hossain, S.: Bangla handwritten character recognition using convolutional neural network with data augmentation. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 318–323. IEEE (2019)
Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015)
Goyal, P., Dollar, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., He, K.: Accurate, large minibatch sgd: training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)
Greenfield, P., Miller, J.T., Hsu, J., White, R.L.: Numarray: a new scientific array package for python. PyCon DC (2003)
Grundland, M., Dodgson, N.A.: Decolorize: fast, contrast enhancing, color to grayscale conversion. Pattern Recogn. 40(11), 2891–2896 (2007)
Gti: Hand gesture recognition database (2018), https://www.kaggle.com/gtiupm/leapgestrecog
Gulli, A., Pal, S.: Deep learning with Keras. Packt Publishing Ltd (2017)
Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 4, 532–550 (1987)
Hossain, M.S., Amin, S.U., Alsulaiman, M., Muhammad, G.: Applying deep learning for epilepsy seizure detection and brain mapping visualization. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 15(1s), 1–17 (2019)
Islam, M.Z., Hossain, M.S., ul Islam, R., Andersson, K.: Static hand gesture recognition using convolutional neural network with data augmentation. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 324–329. IEEE (2019)
Islam, R.U., Hossain, M.S., Andersson, K.: A deep learning inspired belief rulebased expert system. IEEE Access 8, 190637–190651 (2020)
Jalab, H.A.: Static hand gesture recognition for human computer interaction. Inf. Technol. J. 11(9), 1265 (2012)
Kabir, S., Islam, R.U., Hossain, M.S., Andersson, K.: An integrated approach of belief rule base and deep learning to predict air pollution. Sensors 20(7), 1956 (2020)
Nandagopalan, S., Kumar, P.K.: Deep convolutional network based saliency prediction for retrieval of natural images. In: International Conference on Intelligent Computing & Optimization. pp. 487–496. Springer (2018)
Nguyen, T.N., Huynh, H.H., Meunier, J.: Static hand gesture recognition using artificial neural network. J. Image Graph. 1(1), 34–38 (2013)
Nguyen, T.N., Huynh, H.H., Meunier, J.: Static hand gesture recognition using principal component analysis combined with artificial neural network. J. Autom. Control Eng. 3(1), 40–45 (2015)
Oyedotun, O.K., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017)
Oz¨o˘g¨ur-Aky¨uz, S., Otar, B.C., Atas, P.K.: Ensemble cluster pruning via convex- concave programming. Comput. Intell. 36(1), 297–319 (2020)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Stergiopoulou, E., Papamarkos, N.: Hand gesture recognition using a neural network shape fitting technique. Eng. Appl. Artif. Intell. 22(8), 1141–1158 (2009)
Uddin Ahmed, T., Jamil, M.N., Hossain, M.S., Andersson, K., Hossain, M.S.: An integrated real-time deep learning and belief rule base intelligent system to assess facial expression under uncertainty. In: 9th International Conference on Informatics, Electronics & Vision (ICIEV). IEEE Computer Society (2020)
Wang, W., Yang, J., Xiao, J., Li, S., Zhou, D.: Face recognition based on deep learning. In: International Conference on Human Centered Computing. pp. 812– 820. Springer (2014)
Yingxin, X., Jinghua, L., Lichun, W., Dehui, K.: A robust hand gesture recognition method via convolutional neural network. In: 6th International Conference on Digital Home (ICDH), pp. 64–67. IEEE (2016)
Zhu, Y., Huang, C.: An improved median filtering algorithm for image noise reduction. Phys. Procedia 25, 609–616 (2012)
Zisad, S.N., Hossain, M.S., Andersson, K.: Speech emotion recognition in neurological disorders using convolutional neural network. In: International Conference on Brain Informatics. pp. 287–296. Springer (2020)
Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn. Lett. 27(7), 773–780 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Basnin, N., Nahar, L., Hossain, M.S. (2021). An Integrated CNN-LSTM Model for Micro Hand Gesture Recognition. In: Vasant, P., Zelinka, I., Weber, GW. (eds) Intelligent Computing and Optimization. ICO 2020. Advances in Intelligent Systems and Computing, vol 1324. Springer, Cham. https://doi.org/10.1007/978-3-030-68154-8_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-68154-8_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68153-1
Online ISBN: 978-3-030-68154-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)