Abstract
As computers become more pervasive in society, facilitating natural human–computer interaction (HCI) will have a positive impact on their use. Hence, there has been growing interest in the development of new approaches and technologies for bridging the human–computer barrier. The ultimate aim is to bring HCI to a regime where interactions with computers will be as natural as an interaction between humans, and to this end, incorporating gestures in HCI is an important research area. Gestures have long been considered as an interaction technique that can potentially deliver more natural, creative and intuitive methods for communicating with our computers. This paper provides an analysis of comparative surveys done in this area. The use of hand gestures as a natural interface serves as a motivating force for research in gesture taxonomies, its representations and recognition techniques, software platforms and frameworks which is discussed briefly in this paper. It focuses on the three main phases of hand gesture recognition i.e. detection, tracking and recognition. Different application which employs hand gestures for efficient interaction has been discussed under core and advanced application domains. This paper also provides an analysis of existing literature related to gesture recognition systems for human computer interaction by categorizing it under different key parameters. It further discusses the advances that are needed to further improvise the present hand gesture recognition systems for future perspective that can be widely used for efficient human computer interaction. The main goal of this survey is to provide researchers in the field of gesture based HCI with a summary of progress achieved to date and to help identify areas where further research is needed.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
A Forge.NET (2012) http://www.aforgenet.com/framework/
Alijanpour N, Ebrahimnezhad H, Ebrahimi A (2008) Inner distance based hand gesture recognition for devices control. In: International conference on innovations in information technology, pp 742–746
Alon J, Athitsos V, Yuan Q, Sclaroff S (2005) Simultaneous localization and recognition of dynamic hand gestures. In: IEEE workshop on motion and video computing (WACV/MOTION’05), pp 254–260
Alon J, Athitsos V, Yuan Q, Sclaroff S (2009) A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Trans Pattern Analy Mach Intell 31(9): 1685–1699
Alpern M, Minardo K (2003) Developing a car gesture interface for use as a secondary task. In: CHI ’03 extended abstracts on human factors in computing systems. ACM Press, pp 932–933
Andrea C (2001) Dynamic time warping for offline recognition of a small gesture vocabulary. In: Proceedings of the IEEE ICCV workshop on recognition, analysis, and tracking of faces and gestures in real-time systems, July–August, p 83
Appenrodt J, Handrich S, Al-Hamadi A, Michaelis B (2010) Multi stereo camera data fusion for fingertip detection in gesture recognition systems. In: International conference of soft computing and pattern recognition (SoCPaR), 2010, pp 35–40
Argyros A, Lourakis MIA (2004a) Real-time tracking of multiple skin-colored objects with a possibly moving camera. In: Proceedings of the European conference on computer vision, Prague, pp 368–379
Argyros A, Lourakis MIA (2004b) 3D tracking of skin-colored regions by a moving stereoscopic observer. Appl Opt 43(2): 366–378
Argyros A, Lourakis MIA (2006) Binocular hand tracking and reconstruction based on 2D shape matching. In: Proceedings of the international conference on pattern recognition (ICPR), Hong-Kong
Bandera JP, Marfil R, Bandera A, Rodríguez JA, Molina-Tanco L, Sandoval F (2009) Fast gesture recognition based on a two-level representation. Pattern Recogn Lett 30: 1181–1189
Bao J, Song A, Guo Y, Tang H (2011) Dynamic hand gesture recognition based on SURF tracking. In: International conference on electric information and control engineering (ICEICE), pp 338–341
Baxter J (2000) A model of inductive bias learning. J Artif Intell Res 12: 149–198
Bellarbi A, Benbelkacem S, Zenati-Henda N, Belhocine M (2011) Hand gesture interaction using color-based method for Tabletop interfaces. In: IEEE 7th international symposium on intelligent signal processing (WISP), pp 1–6
Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4): 509–522
Berci N, Szolgay P (2007) Vision based human–machine interface via hand gestures. In: 18th European conference on circuit theory and design (ECCTD 2007), pp 496–499
Bergh M, Gool L (2011) Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: Workshop on applications of computer vision (WACV), IEEE, pp 66–72
Bergh MV, Meier EK, Bosch’e F, Gool LV (2009) Haarlet-based hand gesture recognition for 3D interaction, workshop on applications of computer vision (WACV), pp 1–8
Bernardes J, Nakamura R, Tori R (2009) Design and implementation of a flexible hand gesture command interface for games based on computer vision. In: 8th Brazilian symposium on digital games and entertainment, pp 64–73
Berry G (1998) Small-wall, a multimodal human computer intelligent interaction test bed with applications, Dept. of ECE, University of Illinois at Urbana-Champaign, MS thesis
Bhuyan MK, Ghoah D, Bora PK (2006) A framework for hand gesture recognition with applications to sign language. In: Annual IEEE India conference, pp 1–6
Bimbo AD, Landucci L, Valli A (2006) Multi-user natural interaction system based on real-time hand tracking and gesture recognition. In: 18th International conference on pattern recognition (ICPR’06), pp 55–58
Binh ND, Ejima T (2006) A new approach dedicated to hand gesture recognition. In: 5th IEEE international conference on cognitive informatics (ICCI’06), pp 62–67
Birdal A, Hassanpour R (2008) Region based hand gesture recognition. In: 16th International conference in central Europe on computer graphics, visualization and computer vision, pp 1–7
Birk H, Moeslund TB, Madsen CB (1997) Real-time recognition of hand alphabet gestures using principal component analysis. In: Proceedings of the Scandinavian conference on image analysis, Lappeenranta
Blake A, North B, Isard M (1999) Learning multi-class dynamics. In: Proceedings advances in neural information processing systems (NIPS), vol 11, pp 389–395
Bolt RA, Herranz E (1992) Two-handed gesture in multi-modal natural dialog. In: Proceedings of the 5th annual ACM symposium on user interface software and technology, ACM Press, pp 7–14
Boulay B (2007) Human posture recognition for behavior understanding. PhD thesis, Universit’e de Nice-Sophia Antipolis
Bourke A, O’Brien J, Lyons G (2007) Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm. Gait & Posture 26(2):194–199. http://www.sciencedirect.com/science/article/B6T6Y-4MBCJHV-1/2/f87e4f1c82f3f93a3a5692357e3fe00c
Bowden R, Zisserman A, Kadir T, Brady M (2003) Vision based interpretation of natural sign languages. In: Exhibition at ICVS03: the 3rd international conference on computer vision systems. ACM Press, pp 1–2
Bradski G (1998) Real time face and object tracking as a component of a perceptual user interface. In: IEEE workshop on applications of computer vision. Los Alamitos, California, pp 214–219
Bradski G, Kaehler A (2008) Learning OpenCV, O‘Reilly, pp 337–341
Bretzner L, Laptev I, Lindeberg T (2002) Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering. In: Fifth IEEE international conference on automatic face and gesture recognition, pp 405–410. doi:10.1109/AFGR.2002.1004190
Buchmann V, Violich S, Billinghurst M, Cockburn A (2004) Fingartips: gesture based direct manipulation in augmented reality. In: 2nd international conference on computer graphics and interactive techniques, ACM Press, pp 212–221
Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Kluwer, Boston, pp 1–43
Cao X, Balakrishnan R (2003) Visionwand: interaction techniques for large displays using a passive wand tracked in 3d. In: ‘UIST ’03: proceedings of the 16th annual ACM symposium on User Interface software and technology. ACM Press, New York, pp 173–182
Chai D, Ngan K (1998) Locating the facial region of a head and-shoulders color image. In: IEEE international conference on automatic face and gesture recognition, pp 124–129, Piscataway
Chalechale A, Naghdy G (2007) Visual-based human–machine interface using hand gestures. In: 9th International symposium on signal processing and its applications (ISSPA 2007), pp 1–4
Chalechale A, Safaei F, Naghdy G, Premaratn P (2005) Hand gesture selection and recognition for visual-based human–machine interface. In: IEEE international conference on electro information technology, pp 1–6
Chang CC (2006) Adaptive multiple sets of CSS features for hand posture recognition. Neuro Comput 69: 2017–2025
Charniak E (1993) Statistical language learning. MIT Press, Cambridge
Chatty S, Lecoanet P (1996) Pen computing for air traffic control. In: Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press, pp 87–94
Chaudhary A, Raheja JL, Das K, Raheja S (2011) Intelligent approaches to interact with machines using hand gesture recognition in natural way: a survey. Int J Comput Sci Eng Survey (IJCSES) 2(1): 122–133
Chen YT, Tseng KT (2007) Developing a multiple-angle hand gesture recognition system for human machine interactions. In: 33rd annual conference of the IEEE industrial electronics society (IECON), pp 489–492
Chen Q, Georganas ND, Petriu EM (2007) Real-time vision-based hand gesture recognition using Haar-like features. In: Conference on instrumentation and measurement technology (IMTC 2007), pp 1–6
Chen Q, Georganas ND, Petriu M (2008) Hand gesture recognition using Haar-like features and a stochastic context-free grammar. IEEE Trans Instrum Meas 57(8): 1562–1571
Cheng J, Xie X, Bian W, Tao D (2012) Feature fusion for 3D hand gesture recognition by learning a shared hidden space. Pattern Recogn Lett 33: 476–484
Choras RS (2009) Hand shape and hand gesture recognition. In: IEEE symposium on industrial electronics and applications (ISIEA 2009), pp 145–149
Chung WK, Wu X, Xu Y (2009) A real time hand gesture recognition based on Haar wavelet representation. In: International conference on robotics and biomimetics, Bangkok, pp 336–341
Cohen PR, Johnston M, McGee D, Oviatt S, Pittman J, Smith I, Chen L, Clow J (1997) Quickset: multimodal interaction for distributed applications. In: Proceedings of the fifth ACM international conference on Multimedia, ACM Press, pp 31–40
Conci N, Ceresato P, De Natale FGB (2007) Natural human–machine interface using an interactive virtual blackboard. In: IEEE international conference on image processing, pp 181–184
Cootes TF, Taylor CJ (1992) Active shape models smart snakes. In: British machine vision conference, pp 266–275
Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models—their training and applications. Comput Vis Image Underst 61(1): 38–59
Corera S, Krishnarajah N (2011) Capturing hand gesture movement: a survey on tools techniques and logical considerations. In: Proceedings of Chi Sparks 2011 HCI research, innovation and implementation, Arnhem, Netherlands. http://proceedings.chi-sparks.nl/documents/Education-Gestures/FP-35-AC-EG.pdf
Cote M, Payeur P, Comeau G (2006) Comparative study of adaptive segmentation techniques for gesture analysis in unconstrained environments. In: IEEE international workshop on imagining systems and techniques, pp 28–33
Crowley JL, Jolle Coutaz FB (2000) Perceptual user interfaces: things that see. Commun ACM 43(3): 54–64
Crowley J, Berard F, Coutaz J (1995) Finger tracking as an input device for augmented reality. In: International workshop on gesture and face recognition, Zurich
Cui Y, Weng J (1996) Hand sign recognition from intensity image sequences with complex background. In: Proceedings of the IEEE computer vision and pattern recognition (CVPR), pp 88–93
Cui Y, Swets D, Weng J (1995) Learning-based hand sign recognition using shoslf-m. In: International workshop on automatic face and gesture recognition, Zurich, pp 201–206
Cutler R, Turk M (1998) View-based interpretation of real-time optical flow for gesture recognition. In: Proceedings of the international conference on face and gesture recognition. IEEE Computer Society, Washington, pp 416–421
Darrell T, Essa I, Pentland A (1996) Task-specific gesture analysis in real-time using interpolated views. IEEE Trans Pattern Anal Mach Intell 18(12): 1236–1242
Davis JW, Vaks S (2001) A perceptual user interface for recognizing head gesture acknowledgements. In: Proceedings of the 2001 workshop on perceptive user interfaces. ACM Press, pp 1–7
De Tan T, Geo ZM (2011) Research of hand positioning and gesture recognition based on binocular vision. In: EEE international symposium on virtual reality innovation 2011, pp 311–315
Deng LY, Lee DL, Keh HC, Liu YJ (2010) Shape context based matching for hand gesture recognition. In: IET international conference on frontier computing. Theory, technologies and applications, pp 436–444
Derpanis KG (2004) A review of vision-based hand gestures. http://cvr.yorku.ca/members/gradstudents/kosta/publications/file_Gesture_review.pdf
Derpanis KG (2005) Mean shift clustering, Lecture Notes. http://www.cse.yorku.ca/~kosta/CompVis_Notes/mean_shift.pdf
Du H, Xiong W, Wang Z (2011) Modeling and interaction of virtual hand based on virtools. In: International conference on multimedia technology (ICMT), pp 416–419
Eamonn K, Pazzani MJ (2001) Derivative dynamic time warping. In: First international SIAM international conference on data mining, Chicago
Elmezain M, Al-Hamadi A, Michaelis B (2009) Hand trajectory-based gesture spotting and recognition using HMM. In: 16th IEEE international conference on image processing (ICIP 2009), pp 3577–3580
Elmezain M, Al-Hamadi A, Sadek S, Michaelis M (2010) Robust methods for hand gesture spotting and recognition using hidden Markov models and conditional random fields. In: IEEE international symposium on signal processing and information technology (ISSPIT), pp 133–136
EyeSight’s (2012) http://www.eyesight-tech.com/
Eyetoy (2003) http://asia.gamespot.com/eyetoy-play/
Fang G, Gao W, Zhao D (2003) Large vocabulary sign language recognition based on hierarchical decision trees. In: Proceedings of the 5th international conference on multimodal interfaces. ACM Press, pp 125–131
Fang Y, Wang K, Cheng J, Lu H (2007) A real-time hand gesture recognition method. In: IEEE international conference on multimedia and expo, pp 995–998
Ferscha A, Resmerita S, Holzmann C, Reichor M (2005) Orientation sensing for gesture-based interaction with smart artifacts. Comput Commun 28: 1552–1563
Forsberg A, Dieterich M, Zeleznik R (1998) The music notepad. In: Proceedings of the 11th annual ACM symposium on user interface software and technology, ACM Press, pp 203–210
Francois R, Medioni G (1999) Adaptive color background modeling for real-time segmentation of video streams. In: International conference on imaging science, systems, and technology, Las Vegas, pp 227–232
Freeman W, Weissman C (1995) Television control by hand gestures. In: International workshop on automatic face and gesture recognition, Zurich, pp 179–183
Freeman W, Tanaka K, Ohta J, Kyuma K (1996) Computer vision for computer games. In: Proceedings of the second international conference on automatic face and gesture recognition, pp 100–105
Freund Y, Schapire R (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1): 119–139
Friedman J, Hastie T, Tibshiranim R (2000) Additive logistic regression: a statistical view of boosting. Ann Stat 28(2): 337–374
Gandy M, Starner T, Auxier J, Ashbrook D (2000) The gesture pendant: a self illuminating, wearable, infrared computer vision system for home automation control and medical monitoring. In: ‘4th IEEE international symposium on wearable computers, IEEE Computer Society, pp 87–94
Gastaldi G, Pareschi A, Sabatini SP, Solari F, Bisio GM (2005) A man-machine communication system based on the visual analysis of dynamic gestures. In: IEEE international conference on image processing (ICIP 2005), pp 397–400
Gavrila DM, Davis LS (1995) Towards 3-d model-based tracking and recognition of human movement: multi-view approach. In: IEEE international workshop on automatic face- and gesture recognition. IEEE Computer Society, Zurich, pp 272–277
Ge SS, Yang Y, Lee TH (2006) Hand gesture recognition and tracking based on distributed locally linear embedding. In: IEEE conference on robotics, automation and mechatronics, pp 1–6
Ge SS, Yang Y, Lee TH (2008) Hand gesture recognition and tracking based on distributed locally linear embedding. Image Vis Comput 26:1607–1620
GestureTek (2008) http://www.gesturetek.com/
Gorce MDL, Fleet DJ, Paragios N (2011) Model-based 3D hand pose estimation from monocular video. IEEE Trans Pattern Anal Mach Intell 33(9): 1793–1805
Goza SM, Ambrose RO, Diftler MA, Spain IM (2004) Telepresence control of the nasa/darpa robonaut on a mobility platform. In: Conference on human factors in computing systems. ACM Press, pp 623–629
Graetzel C, Fong TW, Grange S, Baur C (2004) A non-contact mouse for surgeon-computer interaction. Technol Health Care 12(3): 245–257
Habib HA, Mufti M (2006) Real time mono vision gesture based virtual keyboard system. IEEE Trans Consumer Electron 52(4):1261–1266
Hackenberg G, McCall R, Broll W (2011) Lightweight palm and finger tracking for real-time 3D gesture control. In: IEEE virtual reality conference (VR), pp 19–26
Hall ET (1973) The silent language. Anchor Books. ISBN-13: 978-0385055499
HandGKET (2011) https://sites.google.com/site/kinectapps/kinect
HandVu (2003) http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html
Hardenberg CV, Berard F (2001) Bare-hand human–computer interaction. Proceedings of the ACM workshop on perceptive user interfaces. ACM Press, pp 113–120
He GF, Kang SK, Song WC, Jung ST (2011) Real-time gesture recognition using 3D depth camera. In: 2nd International conference on software engineering and service science (ICSESS), pp 187–190
Heap T, Hogg D (1996) Towards 3D hand tracking using a deformable model. In: IEEE international conference automatic face and gesture recognition, Killington, pp 140–145
Henia OB, Bouakaz S (2011) 3D Hand model animation with a new data-driven method. In: Workshop on digital media and digital content management, IEEE, pp 72–76
Ho MF, Tseng CY, Lien CC, Huang CL (2011) A multi-view vision- based hand motion capturing system. Pattern Recogn 44: 443–453
Holzmann GJ (1925) Finite state machine: Ebook. http://www.spinroot.com/spin/Doc/Book91_PDF/F1.pdf
Hossain M, Jenkin M (2005) Recognizing hand-raising gestures using HMM. In: 2nd Canadian conference on computer and robot vision (CRV’05), pp 405–412
Howe LW, Wong F, Chekima A (2008) Comparison of hand segmentation methodologies for hand gesture recognition. In: International symposium on information technology (ITSim 2008), pp 1–7
Hsieh CC, Liou DH, Lee D (2010) A real time hand gesture recognition system using motion history image. In: 2nd International conference on signal processing systems (ICSPS), pp 394–398
Hu K, Canavan S, Yin L (2010) Hand pointing estimation for human computer interaction based on two orthogonal-views. In: International conference on pattern recognition 2010, pp 3760–3763
Huang S, Hong J (2011) Moving object tracking system based on camshift and Kalman filter. In: International conference on consumer electronics, communications and networks (CECNet), pp 1423–1426
Huang D, Tang W, Ding Y, Wan T, Wu X, Chen Y (2011a) Motion capture of hand movements using stereo vision for minimally invasive vascular interventions. In: Sixth international conference on image and graphics, pp 737–742
Huang DY, Hu WC, Chang SH (2011b) Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination. Expert Syst Appl 38(5):6031–6042
Iannizzotto G, Villari M, Vita L (2001) Hand tracking for human-computer interaction with gray level visual glove: turning back to the simple way. In: Workshop on perceptive user interfaces, ACM digital library, ISBN 1-58113-448-7
Ibarguren A, Maurtua I, Sierra B (2010) Layered architecture for real time sign recognition: hand gesture and movement. Eng Appl Artif Intell 23: 1216–1228
iGesture (2012) http://www.igesture.org/
Ionescu D, Ionescu B, Gadea C, Islam S (2011a) A multimodal interaction method that combines gestures and physical game controllers. In: Proceedings of 20th international conference on computer communications and networks (ICCCN), IEEE, pp 1–6
Ionescu D, Ionescu B, Gadea C, Islam S (2011b) An intelligent gesture interface for controlling TV sets and set-top boxes. In: 6th IEEE international symposium on applied computational intelligence and informatics, pp 159–164
Isard M, Blake A (1998) Condensation—conditional density propagation for visual tracking. Int J Comput Vis 29(1): 5–28
Joslin C, Sawah AE, Chen Q, Georganas N (2005) Dynamic gesture recognition. In: Conference on instrumentation and measurement technology, pp 1706–1711
Ju SX, Black MJ, Minneman S, Kimber D (1997) Analysis of gesture and action in technical talks for video indexing, Technical report, American Association for Artificial Intelligence. AAAI Technical Report SS-97-03
Juang CF, Ku KC (2005) A recurrent fuzzy network for fuzzy temporal sequence processing and gesture recognition. IEEE Trans Syst Man Cybern Part B Cybern 35(4): 646–658
Juang CF, Ku KC, Chen SK (2005) Temporal hand gesture recognition by fuzzified TSK-type recurrent fuzzy network. In: International joint conference on neural networks, pp 1848–1853
Just A, Marcel S (2009) A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition. Comput Vis Image Underst 113: 532–543
Kalman RE (1960) A new approach to linear filtering and prediction problems. Trans ASME J Basic Eng 82: 35–42
Kampmann M (1998) Segmentation of a head into face, ears, neck and hair for knowledge-based analysis-synthesis coding of video-phone sequences. In: Proceedings of the international conference on image processing (ICIP), vol 2, Chicago, pp 876–880
Kanniche MB (2009) Gesture recognition from video sequences. PhD Thesis, University of Nice
Kanungo T, Mount DM, Netanyahu NS, Piatko CD, Silverman R, Wu AY (2002) An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans Pattern Anal Mach Intell 24(7): 881–892
Kapralos B, Hogue A, Sabri H (2007) Recognition of hand raising gestures for a remote learning application. In: Eight international workshop on image analysis for multimedia interactive services (WIAMIS’07), pp 1–4
Karam M (2006) A framework for research and design of gesture-based human computer interactions. PhD Thesis, University of Southampton
Keogh E, Ratanamahatana CA (2005) Exact indexing of dynamic time warping. Knowl Inf Syst 7(3): 358–386
Kevin NYY, Ranganath S, Ghosh D (2004) Trajectory modeling in gesture recognition using cybergloves and magnetic trackers. In: TENCON 2004. IEEE region 10 conference, pp 571–574
Konrad T, Demirdjian D, Darrell T (2003) Gesture + play: full-body interaction for virtual environments. In: ‘CHI ’03 extended abstracts on human factors in computing systems. ACM Press, pp 620–621
Kurata T, Okuma T, Kourogi M, Sakaue K (2001) The hand mouse: GMM hand-color classification and mean shift track-ing. In: International workshop on recognition, analysis and tracking of faces and gestures in real-time systems, Vancouver, pp 119–124
Kuzmanić A, Zanchi V (2007) Hand shape classification using DTW and LCSS as similarity measures for vision-based gesture recognition system. In: International conference on “Computer as a Tool (EUROCON 2007)”, pp 264–269
Laptev I, Lindeberg T (2001) Tracking of multi-state hand models using particle filtering and a hierarchy of multi-scale image features. In: Proceedings of the sScale-space’01, volume 2106 of Lecture Notes in Computer Science, p 63
Lee DH, Hong KS (2010) Game interface using hand gesture recognition. In: 5th international conference on computer sciences and convergence information technology (ICCIT), pp 1092–1097
Lee H-K, Kim JH (1999) An hmm-based threshold model approach for gesture recognition. IEEE Trans Pattern Anal Mach Intell 21(10): 961–973
Lee J, Kunii TL (1995) Model-based analysis of hand posture. IEEE Comput Graphics Appl 15(5): 77–86
Lee D, Park Y (2009) Vision-based remote control system by motion detection and open finger counting. IEEE Trans Consumer Electron 55(4): 2308–2313
Lenman S, Bretzner L, Thuresson B (2002) Using marking menus to develop command sets for computer vision based hand gesture interfaces. In: Proceedings of the second Nordic conference on human–computer interaction, ACM Press, pp 239–242
Li F, Wechsler H (2005) Open set face recognition using transduction. IEEE Trans Pattern Anal Mach Intell 27(11): 1686–1697
Li S, Zhang H (2004) Multi-view face detection with ^oat-boost. IEEE Trans Pattern Anal Mach Intell 26(9): 1112–1123
Liang R-H, Ouhyoung M (1996) A sign language recognition system using hidden Markov model and context sensitive search. In: Proceedings of the ACM symposium on virtual reality software and technology’96, ACM Press, pp 59–66
Licsar A, Sziranyi T (2005) User-adaptive hand gesture recognition system with interactive training. Image Vis Comput 23: 1102–1114
Lin SY, Lai YC, Chan LW, Hung YP (2010) Real-time 3D model-based gesture tracking for multimedia control. In: International conference on pattern recognition, pp 3822–3825
Liu N, Lovell BC (2005) Hand gesture extraction by active shape models. In: Proceedings of the digital imaging computing: techniques and applications (DICTA 2005), pp 1–6
Liu Y, Zhang P (2009) Vision-based human–computer system using hand gestures. In: International conference on computational intelligence and security, pp 529–532
Liu Y, Gan Z, Sun Y (2008) Static hand gesture recognition and its application based on support vector machines. In: Ninth ACIS international conference on software engineering, artificial intelligence, networking, and parallel/distributed computing, pp 517–521
Lloyd S (1982) Least squares quantization in PCM. IEEE Trans Inf Theory 28(2): 129–137
Lu W-L, Little JJ (2006) Simultaneous tracking and action recognition using the pca-hog descriptor. In: The 3rd Canadian conference on computer and robot vision, 2006. Quebec, pp 6–13
Lumsden J, Brewster S (2003) A paradigm shift: alternative interaction techniques for use with mobile & wearable devices. In: Proceedings of the 2003 conference of the centre for advanced studies conference on collaborative research. IBM Press, pp 197–210
Luo Q, Kong X, Zeng G, Fan J (2008) Human action detection via boosted local motion histograms. Mach Vis Appl. doi:10.1007/s00138-008-0168-5
MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: The proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol 1, pp 281–297
Malassiotis S, Strintzis MG (2008) Real-time hand posture recognition using range data. Image Vis Comput 26: 1027–1037
Mammen JP, Chaudhuri S, Agrawal T (2001) Simultaneous tracking of both hands by estimation of erroneous observations. In: Proceedings of the British machine vision conference (BMVC), Manchester
Martin J, Devin V, Crowley J (1998) Active hand tracking. In: IEEE conference on automatic face and gesture recognition, Nara, Japan, pp 573–578
MATLAB (2012) http://www.mathworks.in/products/matlab/
McNeill D (1992) Hand and mind: what gestures reveal about thought. University Of Chicago Press. ISBN: 9780226561325
Mgestyk (2009) http://www.mgestyk.com/
Microsoft Kinect (2012) http://www.microsoft.com/en-us/kinectforwindows/
Mitra S, Acharya T (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybern (SMC)Part C Appl Rev 37(3): 311–324
Modler P, Myatt T (2008) Recognition of separate hand gestures by time delay neural networks based on multi-state spectral image patterns from cyclic hand movements. In: IEEE international conference on systems, man and cybernetics (SMC 2008), pp 1539–1544
Moeslund T, Granum E (2001) A survey of computer vision based human motion capture. Comput Vis Image Underst 81: 231–268
Moyle M, Cockburn A (2002) Gesture navigation: an alternative ‘back’ for the future. In: Human factors in computing systems, ACM Press, New York, pp 822–823
Murthy GRS, Jadon RS (2010) Hand gesture recognition using neural networks. In: 2nd International advance computing conference (IACC), IEEE, pp 134–138
Nickel K, Stiefelhagen R (2003) Pointing gesture recognition based on 3d-tracking of face, hands and head orientation. In: ICMI ’03: proceedings of the 5th international conference on multimodal interfaces. ACM Press, New York, pp 140–146
Nishikawa A, Hosoi T, Koara K, Negoro D, Hikita A, Asano S, Kakutani H, Miyazaki F, Sekimoto M, Yasui M, Miyake Y, Takiguchi S, Monden M (2003) FAce MOUSe: a novel human-machine interface for controlling the position of a laparoscope. IEEE Trans Robotics Autom 19(5): 825–841
Noury N, Barralon P, Virone G, Boissy P, Hamel M, Rumeau P (2003) A smart sensor based on rules and its evaluation in daily routines. In: Engineering in medicine and biology society, 2003. Proceedings of the 25th annual international conference of the IEEE, vol 4, pp 3286–3289
OMRON (2012) http://www.omron.com/
Ong SCW, Ranganath S, Venkatesh YV (2006) Understanding gestures with systematic variations in movement dynamics. Pattern Recogn 39: 1633–1648
Ongkittikul S, Worrall S, Kondoz A (2008) Two hand tracking using colour statistical model with the K-means embedded particle filter for hand gesture recognition. In: 7th Computer information systems and industrial management applications, pp 201–206
Osawa N, Asai K, Sugimoto YY (2000) Immersive graph navigation using direct manipulation and gestures. In: ACM symposium on virtual reality software and technology. ACM Press, pp 147–152
Ottenheimer HJ (2005) The anthropology of language: an introduction to linguistic anthropology. Wadsworth Publishing. ISBN-13: 978-0534594367
Ou J, Fussell SR, Chen X, Setlock LD, Yang J (2003) Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In: Proceedings of the 5th international conference on Multimodal interfaces. ACM Press, pp 242–249
Paiva A, Andersson G, Hk K, Mourao D, Costa M, Martinho C (2002) SenToy in fantasyA: designing an affective sympathetic interface to a computer game. Pers Ubiquitous Comput 6(5–6):378–389
Pang YY, Ismail NA, Gilbert PLS (2010) A real time vision-based hand gesture interaction. In: Fourth Asia international conference on mathematical/analytical modeling and computer simulation, IEEE, pp 237–242
Pantic M, Nijholt A, Pentland A, Huanag TS (2008) Human-centred intelligent human–computer Interaction (HCI 2): how far are we from attaining it?. Int J Auton Adapt Commun Syst 1: 168–187
Patwardhan KS, Roy SD (2007) Hand gesture modelling and recognition involving changing shapes and trajectories, using a predictive EigenTracker. Pattern Recogn Lett 28: 329–334
Paulraj MP, Yaacob S, Desa H, Hema CR (2008) Extraction of head and hand gesture features for recognition of sign language. In: International conference on electronic design, pp 1–6
Pausch R, Williams RD (1990) Tailor: creating custom user interfaces based on gesture. In: Proceedings of the 3rd annual ACM SIGGRAPH symposium on user interface software and technology. ACM Press, pp 123–134
Pavlovic VI, Sharma R, Huang TS (1997) Visual interpretation of hand gestures for human–computer interaction: a review. Trans Pattern Anal Mach Intell 19(7): 677–695
Perez P, Hue C, Vermaak J, Gangnet M (2002) Color-based probabilistic tracking. In: Procedings of the European conference on computer vision, Copenhagen, pp 661–675
Peterfreund N (1999) Robust tracking of position and velocity with Kalman snakes. IEEE Trans Pattern Anal Mach Intell 10(6): 564–569
Pickering CA (2005) Gesture recognition driver controls. IEE J Comput Control Eng 16(1): 27–40
PointGrab’s (2012) http://www.pointgrab.com/
Prieto A, Bellas F, Duro RJ, López-Peña F (2006) An adaptive visual gesture based interface for human machine interaction in intelligent workspaces. In: IEEE international conference on virtual environments, human–computer interfaces, and measurement systems, pp 43–48
Radkowski R, Stritzke C (2012) Interactive hand gesture-based assembly for augmented reality applications. In: ACHI 2012: the fifth international conference on advances in computer–human interactions, IARIA, pp 303–308
Ramage D (2007) Hidden Markov models fundamentals, Lecture Notes. http://cs229.stanford.edu/section/cs229-hmm.pdf
Rashid O, Al-Hamadi A, Michaelis B (2009) A framework for the integration of gesture and oosture recognition using HMM and SVM. In: IEEE international conference on intelligent computing and intelligent systems (ICIS 2009), pp 572–577
Rautaray SS, Agrawal A (2011) A novel human computer interface based on hand gesture recognition using computer vision techniques. In: International conference on intelligent interactive technologies and multimedia (IITM-2011), pp 292–296
Rautaray SS, Agrawal A (2012) Real time hand gesture recognition system for dynamic applications. Int J UbiComp 3(1): 21–31
Reale MJ, Canavan S, Yin L, Hu K, Hung T (2011) A multi-gesture interaction system using a 3-D Iris disk model for Gaze estimation and an active appearance model for 3-D hand pointing. IEEE Trans Multimed 13(3): 474–486
Rehg J, Kanade T (1994) Digiteyes: vision-based hand tracking for human–computer interaction. In: Workshop on motion of non-rigid and articulated bodies, Austin Texas, pp 16–24
Rehg J, Kanade T (1995) Model-based tracking of self-occluding articulated objects. In: Proceedings of the international conference on computer vision (ICCV), pp 612–617
Ren Y, Zhang F (2009a) Hand gesture recognition based on meb-svm. In: Second international conference on embedded software and systems, IEEE Computer Society, Los Alamitos, pp 344–349
Ren Y, Zhang F (2009b) Hand gesture recognition based on MEB-SVM. In: International conferences on embedded software and systems, pp 344–349
Rodriguez S, Picon A, Villodas A (2010) Robust vision-based hand tracking using single camera for ubiquitous 3D gesture interaction. In: IEEE symposium on 3D user interfaces (3DUI), pp 135–136
Sajjawiso T, Kanongchaiyos P (2011) 3D hand pose modeling from uncalibrate monocular images. In: Eighth international joint conference on computer science and software engineering (JCSSE), pp 177–181
Salinas RM, Carnicer RM, Cuevas FJ, Poyato AC (2008) Depth silhouettes for gesture recognition. Pattern Recogn Lett 29: 319–329
Sangineto E, Cupelli M (2012) Real-time viewpoint-invariant hand localization with cluttered backgrounds. Image Vis Comput 30:26–37
Sawah AE, Joslin C, Georganas ND, Petriu EM (2007) A framework for 3D hand tracking and gesture recognition using elements of genetic programming. In: Fourth Canadian conference on computer and robot vision (CRV’07), pp 495–502
Sawah AE, Georganas ND, Petriu EM (2008) A prototype for 3-D hand tracking and posture estimation. IEEE Trans Instrum Meas 57(8): 1627–1636
Saxe D, Foulds R (1996) Toward robust skin identification in video images. In: IEEE international conference on automatic face and gesture recognition, pp 379–384
Schapire R (2002) The boosting approach to machine learning: an overview. In: MSRI workshop on nonlinear estimation and classification
Schlomer T, Poppinga B, Henze N, Boll S (2008) Gesture recognition with a wii controller. In: TEI ’08: proceedings of the 2nd international conference on Tangible and embedded interaction. ACM, New York, pp 11–14
Schmandt C, Kim J, Lee K, Vallejo G, Ackerman M (2002) Mediated voice communication via mobile ip. In: Proceedings of the 15th annual ACM symposium on User interface software and technology. ACM Press, pp 141–150
Schultz M, Gill J, Zubairi S, Huber R, Gordin F (2003) Bacterial contamination of computer keyboards in a teaching hospital. Infect Control Hosp Epidemiol 4(24): 302–303
Sclaroff S, Betke M, Kollios G, Alon J, Athitsos V, Li R, Magee J, Tian TP (2005) Tracking, analysis, and recognition of human gestures in video. In: 8th International conference on document analysis and recognition, pp 806–810
Segen J, Kumar S (1998a) Gesture VR: vision-based 3d Hand interface for spatial interaction. In: Proceedings of the sixth ACM international conference on multimedia. ACM Press, pp 455–464
Segen J, Kumar S (1998b) Video acquired gesture interfaces for the handicapped. In: Proceedings of the sixth ACM international conference on multimedia. ACM Press, pp 45–48
Segen J, Kumar SS (1999) Shadow gestures: 3D hand pose estimation using a single camera. In: Proceedings of the IEEE computer vision and pattern recognition (CVPR), pp 479–485
Senin P (2008) Dynamic time warping algorithm review, technical report. http://csdl.ics.hawaii.edu/techreports/08-04/08-04.pdf
Sharma R, Huang TS, Pavovic VI, Zhao Y, Lo Z, Chu S, Schulten K, Dalke A, Phillips J, Zeller M, Humphrey W (1996) Speech/gesture interface to a visual computing environment for molecular biologists. In: International conference on pattern recognition (ICPR ’96) volume 7276. IEEE Computer Society, pp 964–968
Shimada N, Shirai Y, Kuno Y, Miura J (1998) Hand gesture estimation and model refinement using monocular camera ambiguity limitation by inequality constraints. In: IEEE international conference on face and gesture recognition, Nara, pp 268–273
Shimizu M, Yoshizuka T, Miyamoto H (2007) A gesture recognition system using stereo vision and arm model fitting. In: International congress series 1301, Elsevier, pp 89–92
Sigal L, Sclaroff S, Athitsos V (2004) Skin color-based video segmentation under time-varying illumination. IEEE Trans Pattern Anal Mach Intell 26(7): 862–877
Smith GM, Schraefel MC (2004) The radial scroll tool: scrolling support for stylus- or touch-based document navigation. In: Proceedings of the 17th annual ACM symposium on User interface software and technology, ACM Press, pp 53–56
SoftKinetic, IISU SDK (2012) http://www.softkinetic.com/Solutions/iisuSDK.aspx
Song L, Takatsuka M (2005) Real-time 3D −nger pointing for an augmented desk. In: Australasian conference on user interface, vol 40. Newcastle, pp 99–108
Sriboonruang Y, Kumhom P, Chamnongthai K (2006) Visual hand gesture interface for computer board game control. In: IEEE tenth international symposium on consumer electronics, pp 1–5
Stan S, Philip C (2004) Fastdtw: toward accurate dynamic time warping in linear time and space. In: KDD workshop on mining temporal and sequential data
Staner AT, Pentland A (1995a) Visual recognition of American sign language using hidden Markov models. Technical Report TR-306, Media Lab, MIT
Starner T, Pentland A (1995b) Real time American sign language recognition from video using hidden Markov models, Technical Report 375, MIT Media Lab
Stotts D, Smith JM, Gyllstrom K (2004a) Facespace: endo- and exo-spatial hypermedia in the transparent video face top. In: 15th ACM conference on hypertext & hypermedia. ACM Press, pp 48–57
Stotts D, Smith JM, Gyllstrom K (2004b) Facespace: endo- and exo-spatial hypermedia in the transparent video facetop. In: Proceedings of the fifteenth ACM conference on hypertext & hypermedia. ACM Press, pp 48–57
Suk H, Sin BK, Lee SW (2008) Robust modeling and recognition of hand gestures with dynamic Bayesian network. In: 19th international conference on pattern recognition, pp 1–4
Suka H, Sin B, Lee S (2010) Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recogn 43: 3059–3072
Swindells C, Inkpen KM, Dill JC, Tory M (2002) That one there! Pointing to establish device identity. In: Proceedings of the 15th annual ACM symposium on user interface software and technology. ACM Press, pp 151–160
Teng X, Wu B, Yu W, Liu C (2005) A hand gesture recognition system based on local linear embedding. J Vis Lang Comput 16: 442–454
Terrillon J, Shirazi M, Fukamachi H, Akamatsu S (2000) Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images. In: Proceedings of the international conference on automatic face and gesture recognition (FG), pp 54–61
Terzopoulos D, Szeliski R (1992) Tracking with Kalman Snakes. MIT Press, Cambridge, pp 3–20
Thirumuruganathan S (2010) A detailed introduction to K-nearest neighbor (KNN) algorithm. http://saravananthirumuruganathan.wordpress.com/2010/05/17/a-detailed-introduction-to-k-nearest-neighbor-knn-algorithm/
Tran C, Trivedi MM (2012) 3-D posture and gesture recognition for interactivity in smart spaces. IEEE Trans Ind Inform 8(1): 178–187
Triesch J, Malsburg C (1996) Robust classification of hand postures against complex background. In: IEEE automatic face and gesture recognition, Killington, pp 170–175
Triesch J, Von der Malsburg C (1998) A gesture interface for human-robot-interaction. In: Proceedings of the international conference on automatic face and gesture recognition (FG). IEEE, Nara, Japan, pp 546–551
Tseng KT, Huang WF, Wu CH (2006) Vision-based finger guessing game in human machine interaction. In: IEEE international conference on robotics and biomimetics, pp 619–624
Utsumi A, Ohya J (1998) Image segmentation for human tracking using sequential-image-based hierarchical adaptation. In: Proceedings IEEE computer vision and pattern recognition (CVPR), pp 911–916
Utsumi A, Ohya J (1999) Multiple-hand-gesture tracking using multiple cameras. In: Proceedings of the IEEE computer vision and pattern recognition (CVPR), Colorado, pp 473–478
Vafadar M, Behrad A (2008) Human hand gesture recognition using spatio-temporal volumes for human–computer interaction. In: International symposium in telecommunications, pp 713–718
Vámossy Z, Tóth A, Benedek B (2007) Virtual hand—hand gesture recognition system. In: 5th International symposium on intelligent systems and informatics, pp 97–102
Várkonyi-Kóczy AR, Tusor B (2011) Human–computer interaction for smart environment applications using fuzzy hand posture and gesture models. IEEE Trans Instrum Meas 60(5): 1505–1514
Varona J, Jaume-i-Capó A, Gonzà àlez J, Perales FJ (2009) Toward natural interaction through visual recognition of body gestures in real-time. Interact Comput 21: 3–10
Verma R, Dev A (2009) Vision based hand gesture recognition using finite state machines and fuzzy logic. In: International conference on ultra modern telecommunications & workshops (ICUMT ’09), pp 1–6
Vilaplana JM, Coronado JL (2006) A neural network model for coordination of hand gesture during reach to grasp. Neural Netw 19:12–30
Viola P, Jones M (2001) Robust real-time object detection. In: IEEE workshop on statistical and computational theories of vision, Vancouver
Visser M, Hopf V (2011) Near and far distance gesture tracking for 3D applications. In: 3DTV conference: the true vision-capture, transmission and display of 3D video (3DTV-CON), pp 1–4
Vo N, Tran Q, Dinh TB, Dinh TB, Nguyen QM (2010) An efficient human–computer interaction framework using skin color tracking and gesture recognition. In: IEEE RIVF international conference on computing and communication technologies, research, innovation and vision for the future (RIVF), pp 1–6
Wachs J, Stern H, Edan Y, Kartoun U (2002) Real-time hand gestures using the fuzzy-C-means Algorithm. In: Proceeding of WAC 2002, Florida
Wachs JP, Stern H, Edan Y (2005) Cluster labeling and parameter estimation for the automated setup of a hand-gesture recognition system. IEEE Trans Syst Man Cybern PART A Syst Humans 35(6): 932–944
Wachs JP, Kolsch M, Stern H, Edan Y (2011) Vision-based hand-gesture applications. Commun ACM 54: 60–71
Wang GW, Zhang C, Zhuang J (2012) An application of classifier combination methods in hand gesture recognition. Mathematical Problems in Engineering Volume 2012, Hindawi Publishing Corporation, pp 1–17. doi:10.1155/2012/346951
Ward DJ, Blackwell AF, MacKay DJC (2000) Dasher—a data entry interface using continuous gestures and language models. In: Proceedings of the 13th annual ACM symposium on user interface software and technology, ACM Press, pp 129–137
Webel S, Keil J, Zoellner M (2008) Multi-touch gestural interaction in x3d using hidden markov models. In: VRST ’08: proceedings of the 2008 ACM symposium on vVirtual reality software and technology. ACM, New York, pp 263–264
Wii Nintendo (2006) http://www.nintendo.com/wii
Wilson A, Shafer S (2003) Xwand: UI for intelligent spaces. In: Proceedings of the conference on Human factors in computing systems. ACM Press, pp 545–552
Wohler C, Anlauf JK (1999) An adaptable time-delay neural-network algorithm for image sequence analysis. IEEE Trans Neural Netw 10(6): 1531–1536
Wu M, Balakrishnan R (2003) Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proceedings of the 16th annual ACM symposium on user interface software and technology. ACM Press, pp 193–202
Wu Y, Huang T (1999a) Vision-based gesture recognition: a review. In: Gesture-based communications in HCI, Lecture Notes in Computer Science, vol 1739. Springer, Berlin
Wu Y, Huang TT (1999b) Capturing human hand motion: a divide-and-conquer approach. In: Proceedings of the international conference on computer vision (ICCV), Greece, pp 606–611
Wu Y, Huang TS (2000) View-independent recognition of hand postures. In: Proceedings of the IEEE computer vision and pattern recognition (CVPR), vol 2. Hilton Head Island, SC, pp 84–94
Wu Y, Lin J, Huang T (2001) Capturing natural hand articulation. In: Proceedings of the international conference on computer vision (ICCV), Vancouver, pp 426–432
Wu Y, Lin J, Huang TS (2005) Analyzing and capturing articulated hand motion in image sequences. IEEE Trans Pattern Anal Mach Intell 27(12): 1910–1922
Xiangyu W, Xiujuan L (2010) The study of moving target tracking based on Kalman CamShift in the video. In: 2nd International conference on information science and engineering (ICISE), pp 1–4
Yang M, Ahuja N (1998) Detecting human faces in color images. In: Proceedings of the international conference on image processing (ICIP), Piscataway, pp 127–130
Yang J, Lu W, Waibel A (1998a) Skin-color modeling and adaptation. In: ACCV, pp 687–694
Yang J, Lu W, Waibel A (1998b) Skin-color modeling and adaptation. In: ACCV, pp 687–694
Yang J, Xu J, Li M, Zhang D, Wang C (2011) A real-time command system based on hand gesture recognition. In: Seventh international conference on natural computation, pp 1588–1592
Yi B, Harris FC Jr, Wang L, Yan Y (2005) Real-time natural hand gestures. Comput Sci Eng IEEE 7(3):92–97
Yi X, Qin S, Kang J (2009) Generating 3D architectural models based on hand motion and gesture. Comput Ind 60:677–685
Yilmaz JA, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38: 13
Yin M, Xie X (2003) Estimation of the fundamental matrix from uncalibrated stereo hand images for 3D hand gesture recognition. Pattern Recogn 36(3): 567–584
Yin J, Han Y, Li J, Cao A (2009) Research on real-time object tracking by improved CamShift. In: International symposium on computer network and multimedia technology, pp 1–4
Yuan Q, Sclaroff S, Athitsos V (1995) Automatic 2D hand tracking in video sequences. In: IEEE workshop on applications of computer vision, pp 250–256
Yuan R, Cheng J, Li P, Chen G, Xie C, Xie Q (2010) View invariant hand gesture recognition using 3D trajectory. In: Proceedings of the 8th world congress on intelligent control and automation, Jinan, pp 6315–6320
Yun L, Peng Z (2009) An automatic hand gesture recognition system based on Viola–Jones method and SVMs. In: Second international workshop on computer science and engineering, pp 72–76
Zabulis X, Baltzakis H, Argyros A (2009) Vision-based Hand gesture recognition for human–computer interaction. In: The Universal Access Handbook. LEA
Zeller M et al (1997) A visual computing environment for very large scale biomolecular modeling. In: Proceedings of the IEEE international conference on application specific systems, architectures and processors (ASAP), Zurich, pp 3–12
Zhao S, Tan W, Wu C, Liu C, Wen S (2009) A Novel interactive method of virtual reality system based on hand gesture recognition. In: Chinese control and decision conference (CCDC ’09), pp 5879–5822
Zhu HM, Pun CM (2010) Movement tracking in real-time hand gesture recognition. In: 9th IEEE/ACIS international conference on computer and information science, pp 241–245
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rautaray, S.S., Agrawal, A. Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43, 1–54 (2015). https://doi.org/10.1007/s10462-012-9356-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-012-9356-9