Skip to main content
SpringerLink
Log in

A Survey on 3D Hand Detection and Tracking Algorithms for Human Computer Interfacing

  • Conference paper
  • First Online:
Intelligent Systems Design and Applications (ISDA 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 717))

Abstract

3D hand detection and tracking algorithms has increased research interests in computer vision, pattern recognition, and human-computer interfacing. It is greatly inspired by the emerging technologies like RGBD camera, depth sensors and processing architecture. Therefore, this paper presents a survey on recent works on 3D hand detection and tracking and their applications as a natural user interface to control the computer with hand movements and gestures. It examines the literature in terms of 1) 3D hand capturing techniques used like RGBD cameras, depth sensors, 2) processing with different image processing and computer vision algorithms and their hardware implementation 3) and applications in human computer interfacing for realization of the system. While the emphasis is on 3D mouse and keyboard, the related findings and future challenges are also discussed for practitioners.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kaminani, S.: Human computer interaction issues with touch screen interfaces in the flight deck. In: 2011 IEEE/AIAA 30th Digital Avionics Systems Conference, pp. 6B4-1. IEEE (2011)

    Google Scholar 

  2. Hutama, W., Harashima, H., Ishikawa, H., Manabe, H.: HMK: head-Mounted-Keyboard for Text Input in Virtual or Augmented Reality. In: The Adjunct Publication of the 34th Annual ACM Symposium on User Interface Software and Technology, pp. 115–117. ACM (2021)

    Google Scholar 

  3. Yadegaripour, M., Hadadnezhad, M., Abbasi, A., Eftekhari, F., Samani, A.: The effect of adjusting screen height and keyboard placement on neck and back discomfort, posture, and muscle activities during laptop work. Int. J. Human-Comput. Interact. 37(5), 459–469 (2021)

    Article  Google Scholar 

  4. Yi, X., Liang, C., Chen, H., Song, J., Yu, C., Shil, Y.: From 2D to 3D: facilitating Single-Finger Mid-Air Typing on Virtual Keyboards with Probabilistic Touch Modeling. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 694-695. IEEE (2022)

    Google Scholar 

  5. Toni, B., Darko, J.: A robust hand detection and tracking algorithm with application to natural user interface. In: 2012 Proceedings of the 35th International Convention MIPRO, pp. 1768-1774. IEEE (2012)

    Google Scholar 

  6. Suarez, J., Murphy, R.R.: Hand gesture recognition with depth images: a review. In: 2012 IEEE RO-MAN: the 21st IEEE International Symposium on Robot and Human Interactive Communication, pp. 411–417. IEEE (2012)

    Google Scholar 

  7. Joo, S.I., Weon, S.H., Choi, H.I.: Real-time depth-based hand detection and tracking. Sci. World J. 2014, 17 (2014)

    Article  Google Scholar 

  8. Ma, X., Peng, J.: Kinect sensor-based long-distance hand gesture recognition and fingertip detection with depth information. J. Sens. 2018, 1–9 (2018)

    Google Scholar 

  9. Das, S.S.: Techniques for estimating the direction of pointing gestures using depth images in the presence of orientation and distance variations from the depth sensor Doctoral dissertation. (2022)

    Google Scholar 

  10. Swaminathan, K., Grunnet-Jepsen, A., Keselman, L.: Intel Corp: Compact, low cost VC SEL projector for high performance stereodepth camera. U.S. Patent 10, 924,638. (2021)

    Google Scholar 

  11. Spektor, E., Mor, Z., Rais, D.: PrimeSense Ltd: integrated processor for 3D mapping. U.S. Patent 8,456,517 (2013)

    Google Scholar 

  12. Tran, D.S., Ho, N.H., Yang, H.J., Baek, E.T., Kim, S.H., Lee, G.: Real-time hand gesture spotting and recognition using RGB-D camera and 3D convolutional neural network. Appl. Sci. 10(2), 722 (2020)

    Article  Google Scholar 

  13. Park, M., Hasan, M.M., Kim, J., Chae, O.: Hand detection and tracking using depth and color information. In: Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV’12), 2, pp. 779–785 (2012)

    Google Scholar 

  14. Liu, F., Du, B., Wang, Q., Wang, Y., Zeng, W.: Hand gesture recognition using kinect via deterministic learning. In: 2017 29th Chinese Control and Decision Conference (CCDC), pp. 2127–2132. IEEE (2017)

    Google Scholar 

  15. Krips, M., Lammert, T., Kummert, A.: FPGA implementation of a neural network for a real-time hand tracking system. In: Proceedings 1st IEEE International Workshop on Electronic Design, Test and Applications, pp. 313–317 IEEE (2002)

    Google Scholar 

  16. Oniga, S., Tisan, A., Mic, D., Buchman, A., Vida-Ratiu, A.: Hand postures recognition system using artificial neural networks implemented in FPGA. In: 2007 30th International Spring Seminar on Electronics Technology (ISSE), pp. 507–512. IEEE (2007)

    Google Scholar 

  17. Hikawa, H., Kaida, K.: Novel FPGA implementation of hand sign recognition system with SOM-Hebb classifier. IEEE Trans. Circuits Syst. Video Technol. 25(1), 153–166 (2014)

    Article  Google Scholar 

  18. Wang, Z.: Hardware implementation for a hand recognition system on FPGA. In: 2015 IEEE 5th International Conference on Electronics Information and Emergency Communication, pp. 34–38. IEEE (2015)

    Google Scholar 

  19. Singh, S., Saurav, S., Saini, R., Mandal, A.S., Chaudhury, S.: FPGA-Based Smart Camera System for Real-Time Automated Video Surveillance. In: Kaushik, B.K., Dasgupta, S., Singh, V. (eds.) VDAT 2017. CCIS, vol. 711, pp. 533–544. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-7470-7_52

    Chapter  Google Scholar 

  20. Singh, S., Saurav, S., Shekhar, C., Vohra, A.: Prototyping an automated video surveillance system using FPGAs. Int. J. Image Graph. Signal Process. 8(8), 37 (2016)

    Article  Google Scholar 

  21. Singh, S., Shekhar, C., Vohra, A.: FPGA-based real-time motion detection for automated video surveillance systems. Electronics 5(1), 10 (2016)

    Article  Google Scholar 

  22. Singh, S., Mandal, A.S., Shekhar, C., Vohra, A.: Real-time implementation of change detection for automated video surveillance system. Int. Sch. Res. Not. 2013, 5 (2013)

    Google Scholar 

  23. Temburu, Y., Datar, M., Singh, S., Malviya, V., Patkar, S.: Real time System Implementation for Stereo 3D Mapping and Visual Odometry. In: 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS) pp. 7–13. IEEE (2020)

    Google Scholar 

  24. Sawant, P., Temburu, Y., Datar, M., Ahmed, I., Shriniwas, V., Patkar, Sachin: Single Storage Semi-Global Matching for Real Time Depth Processing. In: Babu, R.V., Prasanna, M., Namboodiri, Vinay P.. (eds.) NCVPRIPG 2019. CCIS, vol. 1249, pp. 14–31. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-8697-2_2

    Chapter  Google Scholar 

  25. Islam, S., Matin, A., Kibria, H.B.: Hand Gesture Recognition Based Human Computer Interaction to Control Multiple Applications. In: Vasant, P., Zelinka, I., Weber, G.-W. (eds.) ICO 2021. LNNS, vol. 371, pp. 397–406. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93247-3_39

    Chapter  Google Scholar 

  26. Mukherjee, S., Ahmed, S.A., Dogra, D.P., Kar, S., Roy, P.P.: Fingertip detection and tracking for recognition of air-writing in videos. Expert Syst. Appl. 136, 217–229 (2019)

    Article  Google Scholar 

  27. Ghosh, P., Singhee, R., Karmakar, R., Maitra, S., Rai, S., Pal, S.B.: Virtual Keyboard Using Image Processing and Computer Vision. In: Tavares, J.R.S., Dutta, P., Dutta, S., Samanta, Debabrata (eds.) Cyber Intelligence and Information Retrieval. LNNS, vol. 291, pp. 71–79. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-4284-5_7

    Chapter  Google Scholar 

  28. Chhibber, N., Surale, H.B., Matulic, F, Vogel, D.: Typealike: near-Keyboard Hand Postures for Expanded Laptop Interaction. In: Proceedings of the ACM on Human-Computer Interaction, 5(ISS), pp.1–20. ACM (2021)

    Google Scholar 

  29. Raees, M., Ullah, S., Rahman, S.U.: VEN-3DVE: vision based egocentric navigation for 3D virtual environments. Int. J. Interact. Des. Manufact. (IJIDEM) 13(1), 35–45 (2019)

    Article  Google Scholar 

  30. Enkhbat, A., Shih, T.K., Thaipisutikul, T., Hakim, N.L., Aditya, W.: HandKey: an Efficient Hand Typing Recognition using CNN for Virtual Keyboard. In: 2020-5th International Conference on Information Technology (INCIT), pp. 315–319. IEEE (2020)

    Google Scholar 

  31. Chua, S.N., Chin, K.Y., Lim, S.F., Jain, P.: Hand Gesture Control for Human-Computer Interaction with Deep Learning. J. Electr. Eng. Technol. 17(3), pp. 1961–1970 (2022)

    Google Scholar 

  32. Du, H., Charbon, E.: 3D hand model fitting for virtual keyboard system. In: 2007 IEEE Workshop on Applications of Computer Vision (WACV’07), pp. 31–31. IEEE (2007)

    Google Scholar 

  33. Robertson, P., Laddaga, R., Van Kleek, M.: Virtual mouse vision based interface. In: Proceedings of the 9th international conference on Intelligent user interfaces, pp. 177–183 (2004)

    Google Scholar 

  34. Hu, Y., Wang, B., Wu, C., Liu, K.R.: Universal Virtual Keyboard using 60 GHz mmWave Radar. In: 2021 IEEE 7th World Forum on Internet of Things (WF-IoT), pp. 385–390. IEEE (2021)

    Google Scholar 

  35. Miwa, M., Honda, K., Sato, M.: Image Defocus Analysis for Finger Detection on A Virtual Keyboard. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 24–30. IEEE (2021)

    Google Scholar 

  36. Ambrus, A.J., Mohamed, A.N., Wilson, A.D., MOUNT, B.J. Andersen, J.D.: Microsoft Technology Licensing LLC: Touch sensitive user interface (2017)

    Google Scholar 

  37. Devrio, N., Harrison, C.: Disco Band: multiview Depth-Sensing Smartwatch Strap for Hand, Body and Environment Tracking. In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, pp. 1-13. (2022)

    Google Scholar 

  38. Han, Y., Li, Z., Wu, L., Mai, S., Xing, X., Fu, H.Y.: High-Speed Two-Dimensional Spectral-Scanning Coherent LiDAR System Based on Tunable VCSEL. J. Lightwave Technol. 25 Oct 2022

    Google Scholar 

Download references

Acknowledgement

This research has been financially supported by The Analytical Center for the Government of the Russian Federation (Agreement No. 70–2021-00143 dd. 01.11.2021, IGK 000000D730321P5Q0002). Authors acknowledge the technical support and review feedback from AILSIA symposium held in conjunction with the 22nd International Conference on Intelligent Systems Design and Applications (ISDA 2022).

Author information

Authors and Affiliations

  1. Thapar Institute of Engineering and Technology, Patiala, India

    Anu Bajaj

  2. Machine Intelligence Research Labs (MIR Labs), Auburn, USA

    Anu Bajaj & Ajith Abraham

  3. Center for Artificial Intelligence, Innopolis University, Innopolis, Russia

    Ajith Abraham

  4. Guru Jambheshwar University of Science and Technology, Hisar, Haryana, India

    Jimmy Rajpal

Authors
  1. Anu Bajaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jimmy Rajpal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ajith Abraham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anu Bajaj .

Editor information

Editors and Affiliations

  1. Faculty of Computing and Data Science, FLAME University, Pune, Maharashtra, India

    Ajith Abraham

  2. Center for Smart Computing Continuum, Burgenland, Austria

    Sabri Pllana

  3. University of Bari, Bari, Italy

    Gabriella Casalino

  4. University of Jinan, Jinan, Shandong, China

    Kun Ma

  5. Department of Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, India

    Anu Bajaj

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bajaj, A., Rajpal, J., Abraham, A. (2023). A Survey on 3D Hand Detection and Tracking Algorithms for Human Computer Interfacing. In: Abraham, A., Pllana, S., Casalino, G., Ma, K., Bajaj, A. (eds) Intelligent Systems Design and Applications. ISDA 2022. Lecture Notes in Networks and Systems, vol 717. Springer, Cham. https://doi.org/10.1007/978-3-031-35510-3_37

Download citation

Publish with us

Policies and ethics

Search

Navigation