Abstract
Many interactive devices and systems, from smartphones and tablets to smart wearables, video game consoles, ambient displays and interactive surfaces to systems rendering virtual and augmented reality environments, leverage users’ capabilities to communicate and interact with gestures of many kinds: taps, touches, grasps, pinches, head nods, pointing, hand poses, signs and emblems, flicks and swipes, and mid-air movements of the fingers, wrists, arms, legs, and the whole body. These gestures are sensed, modeled, recognized, and interpreted by interactive computer systems to leverage the rich expressiveness of human communicative skills and, consequently, enable natural, intuitive, and fluent means of communication for users in relation to a computer interlocutor. This chapter presents multiple aspects relevant to the design and engineering of gesture-based interaction, including desirable quality properties of gesture input, gesture representation and recognition techniques, gesture analysis methods, and corresponding software tools to assist the design of gesture sets for interactive systems and accessibility aspects of gesture interaction for users with various sensory or motor abilities. Also, this chapter is meant to provide an overview of the large scientific literature available on the topic of gesture-based interaction, pointing the reader to both theoretical and practical aspects and highlighting design challenges, technical solutions, and opportunities for designing and engineering gesture-based interaction with computer systems.
Similar content being viewed by others
References
Ali AX, Morris MR, Wobbrock JO (2018) Crowdsourcing similarity judgments for agreement analysis in end-user elicitation studies. In: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, UIST’18. ACM, New York, pp 177–188. https://doi.org/10.1145/3242587.3242621
Ali AX, Morris MR, Wobbrock JO (2019) Crowdlicit: a system for conducting distributed end-user elicitation and identification studies. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–12. https://doi.org/10.1145/3290605.3300485
Ali A, Ringel Morris M, O Wobbrock J (2021) I am iron man: priming improves the learnability and memorability of user-elicited gestures. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3411764.3445758
Aloba A, Anthony L (2021) Characterizing children’s motion qualities: implications for the design of motion applications for children. In: Proceedings of the 2021 International Conference on Multimodal Interaction, ICMI’21. ACM, New York, pp 229–238. https://doi.org/10.1145/3462244.3479941
Aloba A, Woodward J, Anthony L (2020) FilterJoint: toward an understanding of whole-body gesture articulation. In: Proceedings of the 2020 International Conference on Multimodal Interaction, ICMI’20. ACM, New York, pp 213–221. https://doi.org/10.1145/3382507.3418822
Anderson F, Bischof WF (2013) Learning and performance with gesture guides. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 1109–1118. https://doi.org/10.1145/2470654.2466143
Annett M, Grossman T, Wigdor D, Fitzmaurice G (2011) Medusa: a proximity-aware multi-touch tabletop. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST’11. ACM, New York, pp 337–346. https://doi.org/10.1145/2047196.2047240
Anthony L, Wobbrock JO (2010) A lightweight multistroke recognizer for user interface prototypes. In: Proceedings of Graphics Interface 2010, GI’10. Canadian Information Processing Society, pp 245–252. https://doi.org/10.5555/1839214.1839258
Anthony L, Wobbrock JO (2012) $N-Protractor: a fast and accurate multistroke recognizer. In: Proceedings of Graphics Interface 2012, GI’12. Canadian Information Processing Society. https://doi.org/10.5555/2305276.2305296
Anthony L, Kim Y, Findlater L (2013a) Analyzing user-generated youtube videos to understand touchscreen use by people with motor impairments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 1223–1232. https://doi.org/10.1145/2470654.2466158
Anthony L, Vatavu RD, Wobbrock JO (2013b) Understanding the consistency of users’ pen and finger stroke gesture articulation. In: Proceedings of Graphics Interface 2013, GI’13. Canadian Information Processing Society, Canada, pp 87–94. https://doi.org/10.5555/2532129.2532145
Appert C, Zhai S (2009) Using strokes as command shortcuts: cognitive benefits and toolkit support. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’09. ACM, New York, pp 2289–2298. https://doi.org/10.1145/1518701.1519052
Arefin Shimon SS, Lutton C, Xu Z, Morrison-Smith S, Boucher C, Ruiz J (2016) Exploring non-touchscreen gestures for smartwatches. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI’16. ACM, New York, pp 3822–3833. https://doi.org/10.1145/2858036.2858385
Asadi-Aghbolaghi M, Clapés A, Bellantonio M, Escalante HJ, Ponce-López V, Baró X, Guyon I, Kasaei S, Escalera S (2017) A survey on deep learning based approaches for action and gesture recognition in image sequences. In: Proceedings of the 12th IEEE International Conference on Automatic Face Gesture Recognition, FG’17, pp 476–483. https://doi.org/10.1109/FG.2017.150
Ashbrook D, Starner T (2010) MAGIC: a motion gesture design tool. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’10. ACM, New York, pp 2159–2168. https://doi.org/10.1145/1753326.1753653
Azai T, Otsuki M, Shibata F, Kimura A (2018) Open palm menu: a virtual menu placed in front of the palm. In: Proceedings of the 9th Augmented Human International Conference, AH’18. ACM, New York. https://doi.org/10.1145/3174910.3174929
Bai H, Sasikumar P, Yang J, Billinghurst M (2020) A user study on mixed reality remote collaboration with eye gaze and hand gesture sharing. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI’20. ACM, New York, pp 1–13. https://doi.org/10.1145/3313831.3376550
Bailey SW, Bodenheimer B (2012) A comparison of motion capture data recorded from a vicon system and a microsoft kinect sensor. In: Proceedings of the ACM Symposium on Applied Perception, SAP’12. ACM, New York, p 121. https://doi.org/10.1145/2338676.2338703
Bailly G, Müller J, Lecolinet E (2012) Design and evaluation of finger-count interaction: combining multitouch gestures and menus. Int J Hum Comput Stud 70(10):673–689. https://doi.org/10.1016/j.ijhcs.2012.05.006
Bardot S, Rawat S, Nguyen DT, Rempel S, Zheng H, Rey B, Li J, Fan K, Huang DY, Li W, Irani P (2021) ARO: exploring the design of smart-ring interactions for encumbered hands. In: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction, MobileHCI’21. ACM, New York. https://doi.org/10.1145/3447526.3472037
Bau O, Mackay WE (2008) OctoPocus: a dynamic guide for learning gesture-based command sets. In: Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, UIST’08. ACM, New York, pp 37–46. https://doi.org/10.1145/1449715.1449724
Baudel T, Beaudouin-Lafon M (1993) Charade: remote control of objects using free-hand gestures. Commun ACM 36(7):28–35. https://doi.org/10.1145/159544.159562
Bergström J, Hornbæk K (2019) Human–computer interaction on the skin. ACM Comput Surv 52(4). https://doi.org/10.1145/3332166
Bi X, Li Y, Zhai S (2013) FFitts law: modeling finger touch with fitts’ law. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 1363–1372. https://doi.org/10.1145/2470654.2466180
Bilius LB, Vatavu RD (2021) A multistudy investigation of drivers and passengers’ gesture and voice input preferences for in-vehicle interactions. J Intell Transp Syst 25(2):197–220. https://doi.org/10.1080/15472450.2020.1846127
Bilius LB, Vatavu RD, Marquardt N (2021) Smart vehicle proxemics: a conceptual framework operationalizing proxemics in the context of outside-the-vehicle interactions. In: Proceedings of the 18th IFIP TC 13 International Conference, INTERACT’21. Springer, Berlin/Heidelberg, pp 150–171. https://doi.org/10.1007/978-3-030-85616-8_11
Bobick A, Davis J (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267. https://doi.org/10.1109/34.910878
Bobick AF, Intille SS, Davis JW, Baird F, Pinhanez CS, Campbell LW, Ivanov YA, Schütte A, Wilson A (1999) The kidsroom: a perceptually-based interactive and immersive story environment. Presence: Teleoper Virtual Environ 8(4):369–393. https://doi.org/10.1162/105474699566297
Bolt RA (1980) Put-that-there: voice and gesture at the graphics interface. SIGGRAPH Comput Graph 14(3):262–270. https://doi.org/10.1145/965105.807503
Borah PP, Sorathia K (2019) Natural and intuitive deformation gestures for one-handed landscape mode interaction. In: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI’19. ACM, New York, pp 229–236. https://doi.org/10.1145/3294109.3300996
Cadoz C (1994) Le geste canal de communication homme/machine. Tech Sci Inform 13(1):31–61
Cai Z, Han J, Liu L, Shao L (2017) RGB-D datasets using microsoft kinect or similar sensors: a survey. Multimed Tools Appl 76(3):4313–4355. https://doi.org/10.1007/s11042-016-3374-6
Cao X, Zhai S (2007) Modeling human performance of pen stroke gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’07. ACM, New York, pp 1495–1504. https://doi.org/10.1145/1240624.1240850
Caramiaux B, Donnarumma M, Tanaka A (2015) Understanding gesture expressivity through muscle sensing. ACM Trans Comput-Hum Interact 21(6). https://doi.org/10.1145/2687922
Carrington P, Hurst A, Kane SK (2014) The gest-rest: a pressure-sensitive chairable input pad for power wheelchair armrests. In: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility, ASSETS’14. ACM, New York, pp 201–208. https://doi.org/10.1145/2661334.2661374
Cauchard JR, E JL, Zhai KY, Landay JA (2015) Drone & me: an exploration into natural human-drone interaction. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’15. ACM, New York, pp 361–365. https://doi.org/10.1145/2750858.2805823
Chan L, Hsieh CH, Chen YL, Yang S, Huang DY, Liang RH, Chen BY (2015) Cyclops: wearable and single-piece full-body gesture input devices. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI’15. ACM, New York, pp 3001–3009. https://doi.org/10.1145/2702123.2702464
Chen L, Wang F, Deng H, Ji K (2013) A survey on hand gesture recognition. In: Proceedings of the 2013 International Conference on Computer Sciences and Applications, CSA’13. IEEE Computer Society, pp 313–316. https://doi.org/10.1109/CSA.2013.79
Dang H, Buschek D (2021) GestureMap: supporting visual analytics and quantitative analysis of motion elicitation data by learning 2D embeddings. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3411764.3445765
Delamare W, Janssoone T, Coutrix C, Nigay L (2016) Designing 3D gesture guidance: visual feedback and feedforward design options. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, AVI’16. ACM, New York, pp 152–159. https://doi.org/10.1145/2909132.2909260
Dessì S, Spano LD (2020) DG3: exploiting gesture declarative models for sample generation and online recognition. Proc ACM Hum-Comput Interact 4(EICS). https://doi.org/10.1145/3397870
Dingler T, Rzayev R, Shirazi AS, Henze N (2018) Designing consistent gestures across device types: eliciting rsvp controls for phone, watch, and glasses. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–12. https://doi.org/10.1145/3173574.3173993
Dong Z, Zhang J, Lindeman R, Piumsomboon T (2020) Surface vs motion gestures for mobile augmented reality. In: Symposium on Spatial User Interaction, SUI’20. ACM, New York. https://doi.org/10.1145/3385959.3422694
El Ali A, Kildal J, Lantz V (2012) Fishing or a Z? Investigating the effects of error on mimetic and alphabet device-based gesture interaction. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI’12. ACM, New York, pp 93–100. https://doi.org/10.1145/2388676.2388701
Engelbart DC, English WK (1968) A research center for augmenting human intellect. In: Proceedings of the Fall Joint Computer Conference, Part I, 9–11 Dec, 1968, AFIPS’68 (Fall, part I). ACM, New York, pp 395–410. https://doi.org/10.1145/1476589.1476645
Evans AC, Wobbrock JO, Davis K (2016) Modeling collaboration patterns on an interactive tabletop in a classroom setting. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, CSCW’16. ACM, New York, pp 860–871. https://doi.org/10.1145/2818048.2819972
Felberbaum Y, Lanir J (2018) Better understanding of foot gestures: an elicitation study. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–12. https://doi.org/10.1145/3173574.3173908
Ferstl Y, Neff M, McDonnell R (2021) It’s a match! Gesture generation using expressive parameter matching. In: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS’21. Richland, pp 1495–1497
Fu LP, Landay J, Nebeling M, Xu Y, Zhao C (2018) Redefining natural user interface. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA’18. ACM, New York, pp 1–3. https://doi.org/10.1145/3170427.3190649
Gheran BF, Vatavu RD (2020) From controls on the steering wheel to controls on the finger: using smart rings for in-vehicle interactions. In: Companion Publication of the 2020 ACM Designing Interactive Systems Conference, DIS’20 Companion. ACM, New York, pp 299–304. https://doi.org/10.1145/3393914.3395851
Gheran BF, Vanderdonckt J, Vatavu RD (2018a) Gestures for smart rings: empirical results, insights, and design implications. In: Proceedings of the 2018 Designing Interactive Systems Conference, DIS’18. ACM, New York, pp 623–635. https://doi.org/10.1145/3196709.3196741
Gheran BF, Vatavu RD, Vanderdonckt J (2018b) Ring x2: designing gestures for smart rings using temporal calculus. In: Proceedings of the ACM Conference Companion Publication on Designing Interactive Systems, DIS’18 Companion. ACM, New York, pp 117–122. https://doi.org/10.1145/3197391.3205422
Gil H, Kim H, Oakley I (2018) Fingers and angles: exploring the comfort of touch input on smartwatches. Proc ACM Interact Mob Wearable Ubiquitous Technol 2(4). https://doi.org/10.1145/3287042
Greenberg S, Marquardt N, Ballendat T, Diaz-Marino R, Wang M (2011) Proxemic interactions: the new ubicomp? Interactions 18(1):42–50. https://doi.org/10.1145/1897239.1897250
Gupta A, Irudayaraj A, Chandran V, Palaniappan G, Truong KN, Balakrishnan R (2016) Haptic learning of semaphoric finger gestures. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST’16. ACM, New York, pp 219–226. https://doi.org/10.1145/2984511.2984558
Hayashi E, Lien J, Gillian N, Giusti L, Weber D, Yamanaka J, Bedal L, Poupyrev I (2021) RadarNet: efficient gesture recognition technique utilizing a miniature radar sensor. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3411764.3445367
Hoffman M, Varcholik P, LaViola JJ (2010) Breaking the status quo: improving 3D gesture recognition with spatially convenient input devices. In: Proceedings of the 2010 IEEE Virtual Reality Conference, VR’10. IEEE Computer Society, pp 59–66. https://doi.org/10.1109/VR.2010.5444813
Holz C, Baudisch P (2011) Understanding touch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’11. ACM, New York, pp 2501–2510. https://doi.org/10.1145/1978942.1979308
Hoober S (2015) Fingers, thumbs, and people. Interactions 22(3):48–51. https://doi.org/10.1145/2745957
Hu B, Rakthanmanon T, Hao Y, Evans S, Lonardi S, Keogh E (2015) Using the minimum description length to discover the intrinsic cardinality and dimensionality of time series. Data Min Knowl Discov 29(2):358–399. https://doi.org/10.1007/s10618-014-0345-2
Hu F, He P, Xu S, Li Y, Zhang C (2020) FingerTrak: continuous 3D hand pose tracking by deep learning hand silhouettes captured by miniature thermal cameras on wrist. Proc ACM Interact Mob Wearable Ubiquitous Technol 4(2). https://doi.org/10.1145/3397306
Hu X, Wang J, Gao W, Yu C, Shi Y (2021) FootUI: assisting people with upper body motor impairments to use smartphones with foot gestures on the bed. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3411763.3451782
Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, CHI’97. ACM, New York, pp 234–241. https://doi.org/10.1145/258549.258715
Ishii H, Lakatos D, Bonanni L, Labrune JB (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19(1):38–51. https://doi.org/10.1145/2065327.2065337
Jang S, Stuerzlinger W, Ambike S, Ramani K (2017) Modeling cumulative arm fatigue in mid-air interaction based on perceived exertion and kinetics of arm motion. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’17. ACM, New York, pp 3328–3339. https://doi.org/10.1145/3025453.3025523
Jones B, Sodhi R, Murdock M, Mehra R, Benko H, Wilson A, Ofek E, MacIntyre B, Raghuvanshi N, Shapira L (2014) RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST’14. ACM, New York, pp 637–644. https://doi.org/10.1145/2642918.2647383
Kane SK, Wobbrock JO, Ladner RE (2011) Usable gestures for blind people: understanding preference and performance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’11. ACM, New York, pp 413–422. https://doi.org/10.1145/1978942.1979001
Karam M, schraefel MC (2005) A taxonomy of gestures in human computer interactions. http://eprints.soton.ac.uk/id/eprint/261149
Khurana R, Goel M, Lyons K (2019) Detachable smartwatch: more than a wearable. Proc ACM Interact Mob Wearable Ubiquitous Technol 3(2). https://doi.org/10.1145/3328921
Kin K, Miller T, Bollensdorff B, DeRose T, Hartmann B, Agrawala M (2011) Eden: a professional multitouch tool for constructing virtual organic environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’11. ACM, New York, pp 1343–1352. https://doi.org/10.1145/1978942.1979141
Kin K, Hartmann B, DeRose T, Agrawala M (2012) Proton: multitouch gestures as regular expressions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’12. ACM, New York, pp 2885–2894. https://doi.org/10.1145/2207676.2208694
Koelle M, Ananthanarayan S, Boll S (2020) Social acceptability in HCI: a survey of methods, measures, and design strategies. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI’20. ACM, New York, pp 1–19. https://oi.org/10.1145/3313831.3376162
Kohli L, Whitton M (2005) The haptic hand: providing user interface feedback with the non-dominant hand in virtual environments. In: Proceedings of Graphics Interface 2005, GI’05. Canadian Human-Computer Communications Society, Waterloo, pp 1–8. https://doi.org/10.5555/1089508.1089510
Köpsel A, Bubalo N (2015) Benefiting from legacy bias. Interactions 22(5):44–47. https://doi.org/10.1145/2803169
Kramer J, Burrus N, C DH, Echtler F, Parker M (2012) Hacking the Kinect, 1st edn. Apress. https://doi.org/10.5555/2207813
Kristensson PO, Zhai S (2004) SHARK2: a large vocabulary shorthand writing system for pen-based computers. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, UIST’04. ACM, New York, pp 43–52. https://doi.org/10.1145/1029632.1029640
Krueger MW, Gionfriddo T, Hinrichsen K (1985) VIDEOPLACE-an artificial reality. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’85. ACM, New York, pp 35–40. https://doi.org/10.1145/317456.317463
Kurtenbach GP (1993) The design and evaluation of marking menus. PhD thesis, University of Toronto, Canada. https://doi.org/10.5555/193157, uMI Order No. GAXNN-82896
Kurtenbach G, Hulteen E (1990) Gestures in human-computer communication. In: Laurel B (ed) The art of human-computer interface design. Addison-Wesley Publishing Company, Inc., New York
Lafreniere B, Jonker RT, Santosa S, Parent M, Glueck M, Grossman T, Benko H, Wigdor D (2021) False positives vs. false negatives: the effects of recovery time and cognitive costs on input error preference. In: Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology, UIST’21. ACM, New York. pp 54–68. https://doi.org/10.1145/3472749.3474735
LaViola JJ (1999) A survey of hand posture and gesture recognition techniques and technology. Technical report, Brown University. https://doi.org/10.5555/864649
Lawson JYL, Vanderdonckt J, Vatavu RD (2018) Mass-computer interaction for thousands of users and beyond. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA’18. ACM, New York, pp 1–6. https://doi.org/10.1145/3170427.3188465
Le HV, Kosch T, Bader P, Mayer S, Henze N (2018) PalmTouch: using the palm as an additional input modality on commodity smartphones. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–13. https://doi.org/10.1145/3173574.3173934
Lee JC (2008) Hacking the nintendo wii remote. IEEE Pervasive Comput 7(3):39–45. https://doi.org/10.1109/MPRV.2008.53
Lee JC (2010) In search of a natural gesture. XRDS 16(4):9–12. https://doi.org/10.1145/1764848.1764853
Lee D, Lee Y, Shin Y, Oakley I (2018) Designing socially acceptable hand-to-face input. In: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, UIST’18. ACM, New York, pp 711–723. https://doi.org/10.1145/3242587.3242642
Lefeuvre K, Totzauer S, Storz M, Kurze A, Bischof A, Berger A (2018) Bricks, blocks, boxes, cubes, and dice: on the role of cubic shapes for the design of tangible interactive devices. In: Proceedings of the 2018 Designing Interactive Systems Conference, DIS’18. ACM, New York, pp 485–496. https://doi.org/10.1145/3196709.3196768
Leiva LA, Martín-Albo D, Plamondon R (2015) Gestures à go: authoring synthetic human-like stroke gestures using the kinematic theory of rapid movements. ACM Trans Intell Syst Technol 7(2). https://doi.org/10.1145/2799648
Leiva LA, Martín-Albo D, Plamondon R, Vatavu RD (2018) KeyTime: super-accurate prediction of stroke gesture production times. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–12. https://doi.org/10.1145/3173574.3173813
Leiva LA, Kljun M, Sandor C, Copic Pucihar K (2020a) The wearable radar: sensing gestures through fabrics. In: Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI’20. ACM, New York. https://doi.org/10.1145/3406324.3410720
Leiva LA, Vatavu RD, Martin-Albo D, Plamondon R (2020b) Omnis prædictio: estimating the full spectrum of human performance with stroke gestures. Int J Hum-Comput Stud 142:102466. https://doi.org/10.1016/j.ijhcs.2020.102466
Li Y (2010a) Gesture search: a tool for fast mobile data access. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, UIST’10. ACM, New York, pp 87–96. https://doi.org/10.1145/1866029.1866044
Li Y (2010b) Protractor: a fast and accurate gesture recognizer. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’10. ACM, New York, pp 2169–2172. https://doi.org/10.1145/1753326.1753654
Li Y, Li T, Patel RA, Yang XD, Zhou X (2018) Self-powered gesture recognition with ambient light. In: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, UIST’18. ACM, New York, pp 595–608. https://doi.org/10.1145/3242587.3242635
Li Z, Chan J, Walton J, Benko H, Wigdor D, Glueck M (2021) Armstrong: an empirical examination of pointing at non-dominant arm-anchored UIs in virtual reality. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3411764.3445064
Lien J, Gillian N, Karagozler ME, Amihood P, Schwesig C, Olson E, Raja H, Poupyrev I (2016) Soli: ubiquitous gesture sensing with millimeter wave radar. ACM Trans Graph 35(4). https://doi.org/10.1145/2897824.2925953
Liu C, Chapuis O, Beaudouin-Lafon M, Lecolinet E (2016) Shared interaction on a wall-sized display in a data manipulation task. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI’16. ACM, New York, pp 2075–2086. https://doi.org/10.1145/2858036.2858039
Liu C, Chapuis O, Beaudouin-Lafon M, Lecolinet E (2017) CoReach: cooperative gestures for data manipulation on wall-sized displays. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’17. ACM, New York, pp 6730–6741. https://doi.org/10.1145/3025453.3025594
Liu Z, Vogel D, Wallace JR (2018) Applying the cumulative fatigue model to interaction on large, multi-touch displays. In: Proceedings of the 7th ACM International Symposium on Pervasive Displays, PerDis’18. ACM, New York. https://doi.org/10.1145/3205873.3205890
Long AC, Landay JA, Rowe LA, Michiels J (2000) Visual similarity of pen gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’00. ACM, New York, pp 360–367. https://doi.org/10.1145/332040.332458
Lü H, Li Y (2012) Gesture coder: a tool for programming multi-touch gestures by demonstration. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’12. ACM, New York, pp 2875–2884. https://doi.org/10.1145/2207676.2208693
Lü H, Li Y (2013) Gesture studio: authoring multi-touch interactions through demonstration and declaration. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, pp 257–266. https://doi.org/10.1145/2470654.2470690
Luo Y, Wu K, Palacios T, Matusik W (2021) KnitUI: fabricating interactive and sensing textiles with machine knitting. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3411764.3445780
Magrofuoco N, Vanderdonckt J (2019) Gelicit: a cloud platform for distributed gesture elicitation studies. Proc ACM Hum-Comput Interact 3(EICS). https://doi.org/10.1145/3331148
Magrofuoco N, Roselli P, Vanderdonckt J (2021) Two-dimensional stroke gesture recognition: a survey. ACM Comput Surv 54(7). https://doi.org/10.1145/3465400
Malu M, Chundury P, Findlater L (2018) Exploring accessible smartwatch interactions for people with upper body motor impairments. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–12. https://doi.org/10.1145/3173574.3174062
Marquardt N, Diaz-Marino R, Boring S, Greenberg S (2011) The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST’11. ACM, New York, pp 315–326. https://doi.org/10.1145/2047196.2047238
Masson D, Goguey A, Malacria S, Casiez G (2017) WhichFingers: identifying fingers on touch surfaces and keyboards using vibration sensors. In: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, UIST’17. ACM, New York, pp 41–48. https://doi.org/10.1145/3126594.3126619
Matthies DJC, Perrault ST, Urban B, Zhao S (2015) Botential: localizing on-body gestures by measuring electrical signatures on the human skin. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI’15. ACM, New York, pp 207–216. https://doi.org/10.1145/2785830.2785859
May KR, Gable TM, Walker BN (2017) Designing an in-vehicle air gesture set using elicitation methods. In: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI’17. ACM, New York, pp 74–83. https://doi.org/10.1145/3122986.3123015
McNeill D (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, USA
Microsoft Corporation (2013) Human interface guidelines v1.6. https://download.microsoft.com/ download/B/0/7/B070724E-52B4-4B1A-BD1B-05CC28D07899/Human_Interface_Guidelines _v1.7.0.pdf
Mitra S, Acharya T (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybernet Part C (Appl Rev) 37(3):311–324. https://doi.org/10.1109/TSMCC.2007.893280
Morris MR (2012) Web on the wall: insights from a multimodal interaction elicitation study. In: Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces, ITS’12. ACM, New York, pp 95–104. https://doi.org/10.1145/2396636.2396651
Morris MR, Huang A, Paepcke A, Winograd T (2006) Cooperative gestures: multi-user gestural interactions for co-located groupware. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’06. ACM, New York, pp 1201–1210. https://doi.org/10.1145/1124772.1124952
Morris MR, Wobbrock JO, Wilson AD (2010) Understanding users’ preferences for surface gestures. In: Proceedings of Graphics Interface 2010, GI’10. Canadian Information Processing Society, Canada, pp 261–268. https://doi.org/10.5555/1839214.1839260
Morris MR, Danielescu A, Drucker S, Fisher D, Lee B, schraefel MC, Wobbrock JO (2014) Reducing legacy bias in gesture elicitation studies. Interactions 21(3):40–45. https://doi.org/10.1145/2591689
Mott ME, Vatavu RD, Kane SK, Wobbrock JO (2016) Smart touch: improving touch accuracy for people with motor impairments with template matching. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI’16. ACM, New York, pp 1934–1946. https://doi.org/10.1145/2858036.2858390
Nacenta MA, Kamber Y, Qiang Y, Kristensson PO (2013) Memorability of pre-designed and user-defined gesture sets. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 1099–1108. https://doi.org/10.1145/2470654.2466142
Nebeling M, Ott D, Norrie MC (2015) Kinect analysis: a system for recording, analysing and sharing multimodal interaction elicitation studies. In: Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS’15. ACM, New York, pp 142–151. https://doi.org/10.1145/2774225.2774846
Norman DA (2010) Natural user interfaces are not natural. Interactions 17(3):6–10. https://doi.org/10.1145/1744161.1744163
Norman DA, Nielsen J (2010) Gestural Interfaces: a step backward in usability. Interactions 17(5):46–49. https://doi.org/10.1145/1836216.1836228
Olwal A, Starner T, Mainini G (2020) E-textile microinteractions: augmenting twist with flick, slide and grasp gestures for soft electronics. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI’20. ACM, New York, pp 1–13. https://doi.org/10.1145/3313831.3376236
Ousmer M, Vanderdonckt J, Buraga S (2019) An ontology for reasoning on body-based gestures. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS’19. ACM, New York. https://doi.org/10.1145/3319499.3328238
Palipana S, Salami D, Leiva LA, Sigg S (2021) Pantomime: mid-air gesture recognition with sparse millimeter-wave radar point clouds. Proc ACM Interact Mob Wearable Ubiquitous Technol 5(1). https://doi.org/10.1145/3448110
Perrault ST, Lecolinet E, Eagan J, Guiard Y (2013) Watchit: simple gestures and eyes-free interaction for wristwatches and bracelets. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 1451–1460. https://doi.org/10.1145/2470654.2466192
Piumsomboon T, Clark A, Billinghurst M, Cockburn A (2013) User-defined gestures for augmented reality. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA’13. ACM, New York, pp 955–960. https://doi.org/10.1145/2468356.2468527
Plamondon R (1995) A kinematic theory of rapid human movements. Part I: movement representation and control. Biol Cybern 72(4):295–307. https://doi.org/doi.org/10.1007/bf00202785
Plaumann K, Lehr D, Rukzio E (2016) Who has the force? Solving conflicts for multi user mid-air gestures for TVs. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, TVX’16. ACM, New York, pp 25–29. https://doi.org/10.1145/2932206.2932208
Rakthanmanon T, Campana B, Mueen A, Batista G, Westover B, Zhu Q, Zakaria J, Keogh E (2012) Searching and mining trillions of time series subsequences under dynamic time warping. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’12. ACM, New York, pp 262–270. https://doi.org/10.1145/2339530.2339576
Rateau H, Lank E, Liu Z (2022) Leveraging smartwatch and earbuds gesture capture to support wearable interaction. Proc ACM Hum-Comput Interact 6(ISS). https://doi.org/10.1145/3567710
Rekik Y, Vatavu RD, Grisoni L (2014a) Match-up & conquer: a two-step technique for recognizing unconstrained bimanual and multi-finger touch input. In: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, AVI’14. ACM, New York, pp 201–208. https://doi.org/10.1145/2598153.2598167
Rekik Y, Vatavu RD, Grisoni L (2014b) Understanding users’ perceived difficulty of multi-touch gesture articulation. In: Proceedings of the 16th International Conference on Multimodal Interaction, ICMI’14. ACM, New York, pp 232–239. https://doi.org/10.1145/2663204.2663273
Rendl C, Greindl P, Probst K, Behrens M, Haller M (2014) Presstures: exploring pressure-sensitive multi-touch gestures on trackpads. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’14. ACM, New York, pp 431–434. https://doi.org/10.1145/2556288.2557146
Rico J, Brewster S (2010) Usable gestures for mobile interfaces: evaluating social acceptability. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’10. ACM, New York, pp 887–896. https://doi.org/10.1145/1753326.1753458
Roy Q, Malacria S, Guiard Y, Lecolinet E, Eagan J (2013) Augmented letters: mnemonic gesture-based shortcuts. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 2325–2328. https://doi.org/10.1145/2470654.2481321
Rubine D (1991) Specifying gestures by example. In: Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH’91. ACM, New York, pp 329–337. https://doi.org/10.1145/122718.122753
Ruiz J, Vogel D (2015) Soft-constraints to reduce legacy and performance bias to elicit whole-body gestures with low arm fatigue. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI’15. ACM, New York, pp 3347–3350. https://doi.org/10.1145/2702123.2702583
Ruiz J, Li Y, Lank E (2011) User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’11. ACM, New York, pp 197–206. https://doi.org/10.1145/1978942.1978971
Saffer D (2009) Designing gestural interfaces. O’Reilly Media, Inc., Sebastopol
Saponas TS, Harrison C, Benko H (2011) Pockettouch: through-fabric capacitive touch input. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST’11. ACM, New York, pp 303–308. https://doi.org/10.1145/2047196.2047235
Schiettecatte B, Vanderdonckt J (2008) AudioCubes: a distributed cube tangible interface based on interaction range for sound design. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, TEI’08. ACM, New York, pp 3–10. https://doi.org/10.1145/1347390.1347394
Schipor OA, Vatavu RD, Wu W (2019) SAPIENS: towards software architecture to support peripheral interaction in smart environments. Proc ACM Hum-Comput Interact 3(EICS). https://doi.org/10.1145/3331153
Schlömer T, Poppinga B, Henze N, Boll S (2008) Gesture recognition with a wii controller. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, TEI’08. ACM, New York, pp 11–14. https://doi.org/10.1145/1347390.1347395
Schönauer C, Mossel A, Zaiundefinedi IA, Vatavu RD (2022) Touch, movement and vibration: user perception of vibrotactile feedback for touch and mid-air gestures. In: Proceedings of Human-Computer Interaction – INTERACT 2015. Springer, Berlin/Heidelberg, pp 165–172. https://doi.org/10.1007/978-3-319-22723-8_14
Schweigert R, Leusmann J, Hagenmayer S, Weiß M, Le HV, Mayer S, Bulling A (2019) KnuckleTouch: enabling knuckle gestures on capacitive touchscreens using deep learning. In: Proceedings of Mensch Und Computer 2019, MuC’19. ACM, New York, pp 387–397. https://doi.org/10.1145/3340764.3340767
Sharma A, Roo JS, Steimle J (2019) Grasping microgestures: eliciting single-hand microgestures for handheld objects. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–13. https://doi.org/10.1145/3290605.3300632
Shilkrot R, Huber J, Steimle J, Nanayakkara S, Maes P (2015) Digital digits: a comprehensive survey of finger augmentation devices. ACM Comput Surv 48(2). https://doi.org/10.1145/2828993
Sluÿters A, Lambot S, Vanderdonckt J (2022) Hand gesture recognition for an off-the-shelf radar by electromagnetic modeling and inversion. In: Proceedings of the 27th International Conference on Intelligent User Interfaces, IUI’22. ACM, New York, pp 506–522. https://doi.org/10.1145/3490099.3511107
Soliman M, Mueller F, Hegemann L, Roo JS, Theobalt C, Steimle J (2018) FingerInput: capturing expressive single-hand thumb-to-finger microgestures. In: Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces, ISS’18. ACM, New York, pp 177–187. https://doi.org/10.1145/3279778.3279799
Stößel C (2009) Familiarity as a factor in designing finger gestures for elderly users. In: Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI’09. ACM, New York. https://doi.org/10.1145/1613858.1613950
Suarez J, Murphy RR (2012) Hand gesture recognition with depth images: a review. In: Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication, pp 411–417. https://doi.org/10.1109/ROMAN.2012.6343787
Sutherland IE (1963) Sketchpad: a man-machine graphical communication system. In: Proceedings of the Spring Joint Computer Conference, AFIPS’63 (Spring), 21–23 May 1963. ACM, New York, pp 329–346. https://doi.org/10.1145/1461551.1461591
Tanase CA, Vatavu RD, Ştefan Gheorghe Pentiuc, Graur A (2008) Detecting and tracking multiple users in the proximity of interactive tabletops. Adv Electr Comput Eng 8(2):61–64
Taranta EM, LaViola JJ (2015) Penny pincher: a blazing fast, highly accurate $-family recognizer. In: Proceedings of the 41st Graphics Interface Conference, GI’15. Canadian Information Processing Society, pp 195–202
Taranta II EM, Samiei A, Maghoumi M, Khaloo P, Pittman CR, LaViola JJ Jr (2017) Jackknife: a reliable recognizer with few samples and many modalities. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’17. ACM, New York, pp 5850–5861. https://doi.org/10.1145/3025453.3026002
Tsandilas T (2018) Fallacies of agreement: a critical review of consensus assessment methods for gesture elicitation. ACM Trans Comput-Hum Interact 25(3). https://doi.org/10.1145/3182168
Tu H, Ren X, Zhai S (2012) A comparative evaluation of finger and pen stroke gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’12. ACM, New York, pp 1287–1296. https://doi.org/10.1145/2207676.2208584
Ungurean OC, Vatavu RD (2022) I gave up wearing rings: insights on the perceptions and preferences of wheelchair users for interactions with wearables. IEEE Pervasive Comput 21(3):92–101. https://doi.org/10.1109/MPRV.2022.3155952
Ungurean OC, Vatavu RD, Leiva LA, Martín-Albo D (2018a) Predicting stroke gesture input performance for users with motor impairments. In: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, MobileHCI’18. ACM, New York, pp 23–30. https://doi.org/10.1145/3236112.3236116
Ungurean OC, Vatavu RD, Leiva LA, Plamondon R (2018b) Gesture input for users with motor impairments on touchscreens: empirical results based on the kinematic theory. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA’18. ACM, New York, pp 1–6. https://doi.org/10.1145/3170427.3188619
Vanderdonckt J, Roselli P, Pérez-Medina JL (2018) !FTL, an articulation-invariant stroke gesture recognizer with controllable position, scale, and rotation invariances. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI’18. ACM, New York, pp 125–134. https://doi.org/10.1145/3242969.3243032
Vanderdonckt J, Magrofuoco N, Kieffer S, Pérez J, Rase Y, Roselli P, Villarreal S (2019) Head and shoulders gestures: exploring user-defined gestures with upper body. In: Proceedings of the International Conference on Human-Computer Interaction, HCII’19. Lecture Notes in Computer Science, vol 11584, pp 192–213. https://doi.org/10.1007/978-3-030-23541-3_15
Vatavu RD, Anthony L, Wobbrock JO (2018a) $Q: a super-quick, articulation-invariant stroke-gesture recognizer for low-resource devices. In: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI’18. ACM, New York. https://doi.org/10.1145/3229434.3229465
Vatavu RD, Gheran BF, Schipor MD (2018b) The impact of low vision on touch-gesture articulation on mobile devices. IEEE Pervasive Comput 17(1):27–37. https://doi.org/10.1109/MPRV.2018.011591059
Velloso E, Schmidt D, Alexander J, Gellersen H, Bulling A (2015) The feet in human–computer interaction: a survey of foot-based interaction. ACM Comput Surv 48(2). https://doi.org/10.1145/2816455
Villarreal-Narvaez S, Vanderdonckt J, Vatavu RD, Wobbrock JO (2020) A systematic review of gesture elicitation studies: what can we learn from 216 studies? In: Proceedings of the 2020 ACM Designing Interactive Systems Conference, DIS’20. Association for Computing Machinery, New York, pp 855–872. https://doi.org/10.1145/3357236.3395511
Vatavu RD (2012) User-defined gestures for free-hand tv control. In: Proceedings of the 10th European Conference on Interactive TV and Video, EuroITV’12. ACM, New York, pp 45–48. https://doi.org/10.1145/2325616.2325626
Vatavu RD (2013) The impact of motion dimensionality and bit cardinality on the design of 3D gesture recognizers. Int J Hum-Comput Stud 71(4):387–409. https://doi.org/10.1016/j.ijhcs.2012.11.005
Vatavu RD (2017a) Beyond features for recognition: human-readable measures to understand users’ whole-body gesture performance. Int J Hum-Comput Interact 33(9):713–730. https://doi.org/10.1080/10447318.2017.1278897
Vatavu RD (2017b) Improving gesture recognition accuracy on touch screens for users with low vision. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’17. ACM, New York, pp 4667–4679. https://doi.org/10.1145/3025453.3025941
Vatavu RD (2017c) Smart-pockets: body-deictic gestures for fast access to personal data during ambient interactions. Int J Hum-Comput Stud 103:1–21. https://doi.org/10.1016/j.ijhcs.2017.01.005
Vatavu RD (2019) The dissimilarity-consensus approach to agreement analysis in gesture elicitation studies. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–13. https://doi.org/10.1145/3290605.3300454
Vatavu RD, Bilius LB (2021) GestuRING: a web-based tool for designing gesture input with rings, ring-like, and ring-ready devices. In: Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology, UIST’21. ACM, New York, pp 710–723. https://doi.org/10.1145/3472749.3474780
Vatavu RD, Ungurean OC (2019) Stroke-gesture input for people with motor impairments: empirical results & research roadmap. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–14. https://doi.org/10.1145/3290605.3300445
Vatavu RD, Ungurean OC (2022) Understanding gesture input articulation with upper-body wearables for users with upper-body motor impairments. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI’22. ACM, New York, pp 2:1–2:16. https://doi.org/10.1145/3491102.3501964
Vatavu RD, Wobbrock JO (2015) Formalizing agreement analysis for elicitation studies: new measures, significance test, and toolkit. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI’15. ACM, New York, pp 1325–1334. https://doi.org/10.1145/2702123.2702223
Vatavu RD, Wobbrock JO (2016) Between-subjects elicitation studies: formalization and tool support. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI’16. ACM, New York, pp 3390–3402. https://doi.org/10.1145/2858036.2858228
Vatavu RD, Wobbrock JO (2022) Clarifying agreement calculations and analysis for end-user elicitation studies. ACM Trans Comput-Hum Interact 29(1). https://doi.org/10.1145/3476101
Vatavu RD, Zaiti IA (2013) Automatic recognition of object size and shape via user-dependent measurements of the grasping hand. Int J Hum-Comput Stud 71(5):590–607. https://doi.org/10.1016/j.ijhcs.2013.01.002
Vatavu RD, Vogel D, Casiez G, Grisoni L (2011) Estimating the perceived difficulty of pen gestures. In: Proceedings of the 13th IFIP TC 13 International Conference on Human-Computer Interaction – Volume Part II, INTERACT’11. Springer, Berlin/Heidelberg, pp 89–106. https://doi.org/10.1007/978-3-642-23771-3_9
Vatavu RD, Anthony L, Wobbrock JO (2012) Gestures as point clouds: a $P recognizer for user interface prototypes. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI’12. ACM, New York, pp 273–280. https://doi.org/10.1145/2388676.2388732
Vatavu RD, Anthony L, Wobbrock JO (2013) Relative accuracy measures for stroke gestures. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, ICMI’13. ACM, New York, pp 279–286. https://doi.org/10.1145/2522848.2522875
Vatavu RD, Anthony L, Wobbrock JO (2014) Gesture heatmaps: understanding gesture performance with colorful visualizations. In: Proceedings of the 16th International Conference on Multimodal Interaction, ICMI’14. ACM, New York, pp 172–179. https://doi.org/10.1145/2663204.2663256
Vatavu RD, Cramariuc G, Schipor DM (2015) Touch interaction for children aged 3 to 6 years: experimental findings and relationship to motor skills. Int J Hum-Comput Stud 74:54–76. https://doi.org/10.1016/j.ijhcs.2014.10.007
Vogel D, Balakrishnan R (2004) Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, UIST’04. ACM, New York, pp 137–146. https://doi.org/10.1145/1029632.1029656
Vogiatzidakis P, Koutsabasis P (2018) Gesture elicitation studies for mid-air interaction: a review. Multimodal Technol Interact 2(4). https://doi.org/10.3390/mti2040065
Walter R, Bailly G, Müller J (2013) StrikeAPose: revealing mid-air gestures on public displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’13. ACM, New York, pp 841–850. https://doi.org/10.1145/2470654.2470774
Wang Y, Neff M (2013) Data-driven glove calibration for hand motion capture. In: Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA’13. ACM, New York, pp 15–24. https://doi.org/10.1145/2485895.2485901
Wang S, Song J, Lien J, Poupyrev I, Hilliges O (2016) Interacting with soli: exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST’16. ACM, New York, pp 851–860. https://doi.org/10.1145/2984511.2984565
Wang R, Yu C, Yang XD, He W, Shi Y (2019) EarTouch: facilitating smartphone use for visually impaired people in mobile and public scenarios. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–13. https://doi.org/10.1145/3290605.3300254
Wang T, Qian X, He F, Hu X, Cao Y, Ramani K (2021) GesturAR: an authoring system for creating freehand interactive augmented reality applications. In: The 34th Annual ACM Symposium on User Interface Software and Technology, UIST’21. ACM, New York, pp 552–567. https://doi.org/10.1145/3472749.3474769
Webb A (2002) Statistical pattern recognition, 2nd edn. John Wiley & Sons, West Sussex
Wellner P (1993) Interacting with paper on the digitaldesk. Commun ACM 36(7):87–96. https://doi.org/10.1145/159544.159630
Wexelblat A (1995) An approach to natural gesture in virtual environments. ACM Trans Comput-Hum Interact 2(3):179–200. https://doi.org/10.1145/210079.210080
Wigdor D, Wixon D (2011) Brave NUI World: designing natural user interfaces for touch and gesture, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco
Wilson A, Bobick A (1999) Parametric hidden Markov models for gesture recognition. IEEE Trans Pattern Anal Mach Intell 21(9):884–900. https://doi.org/10.1109/34.790429
Wobbrock JO, Aung HH, Rothrock B, Myers BA (2005) Maximizing the guessability of symbolic input. In: CHI’05 Extended Abstracts on Human Factors in Computing Systems, CHI EA’05. ACM, New York, Pp 1869–1872. https://doi.org/10.1145/1056808.1057043
Wobbrock JO, Wilson AD, Li Y (2007) Gestures without Libraries, Toolkits or Training: A $1 Recognizer for user interface prototypes. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, UIST’07. ACM, New York, pp 159–168. https://doi.org/10.1145/1294211.1294238
Wobbrock JO, Morris MR, Wilson AD (2009) User-defined gestures for surface computing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’09. ACM, New York, pp 1083–1092. https://doi.org/10.1145/1518701.1518866
Wobbrock JO, Gajos KZ, Kane SK, Vanderheiden GC (2018) Ability-based design. Commun ACM 61(6):62–71. https://doi.org/10.1145/3148051
Wolf K, Schleicher R, Kratz S, Rohs M (2013) Tickle: a surface-independent interaction technique for grasp interfaces. In: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, TEI’13. ACM, New York, pp 185–192. https://doi.org/10.1145/2460625.2460654
Wu M, Balakrishnan R (2003) Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology, UIST’03. ACM, New York, pp 193–202. https://doi.org/10.1145/964696.964718
Wu E, Yuan Y, Yeo HS, Quigley A, Koike H, Kitani KM (2020a) Back-hand-pose: 3D hand pose estimation for a wrist-worn camera via dorsum deformation network. In: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, UIST’20. ACM, New York, pp 1147–1160. https://doi.org/10.1145/3379337.3415897
Wu TY, Qi S, Chen J, Shang M, Gong J, Seyed T, Yang XD (2020b) Fabriccio: touchless gestural input on interactive fabrics. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI’20. ACM, New York, pp 1–14. https://doi.org/10.1145/3313831.3376681
Xia H, Glueck M, Annett M, Wang M, Wigdor D (2022) Iteratively designing gesture vocabularies: a survey and analysis of best practices in the hci literature. ACM Trans Comput-Hum Interact 29(4). https://doi.org/10.1145/3503537
Xiao R, Laput G, Harrison C (2014) Expanding the input expressivity of smartwatches with mechanical pan, twist, tilt and click. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’14. ACM, New York, pp 193–196. https://doi.org/10.1145/2556288.2557017
Yan Y, Yu C, Ma X, Yi X, Sun K, Shi Y (2018a) VirtualGrasp: leveraging experience of interacting with physical objects to facilitate digital object retrieval. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1–13. https://doi.org/10.1145/3173574.3173652
Yan Y, Yu C, Yi X, Shi Y (2018b) HeadGesture: hands-free input approach leveraging head movements for HMD devices. Proc ACM Interact Mob Wearable Ubiquitous Technol 2(4). https://doi.org/10.1145/3287076
Yasen M, Jusoh S (2019) A systematic review on hand gesture recognition techniques, challenges and applications. Peer J Comput Sci 5(e218). https://doi.org/10.7717/peerj-cs.218
You CW, Lin YF, Luo E, Lin HY, Kao HLC (2019) Understanding social perceptions towards interacting with on-skin interfaces in public. In: Proceedings of the 23rd International Symposium on Wearable Computers, ISWC’19. ACM, New York, pp 244–253. https://doi.org/10.1145/3341163.3347751
Zaiti IA, Pentiuc SG, Vatavu RD (2015) On free-hand TV control: experimental results on user-elicited gestures with leap motion. Personal and Ubiquitous Computing 19(5-6):821–838. https://doi.org/10.1007/s00779-015-0863-y
Zhai S, Kristensson P, Appert C, Andersen T, Cao X (2012) Foundational issues in touch-surface stroke gesture design: an integrative review. Found Trends Hum-Comput Interact 5(2):97–205. https://doi.org/10.1561/1100000012
Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10. https://doi.org/10.1109/MMUL.2012.24
Zheng J, Vogel D (2016) Finger-aware shortcuts. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI’16. ACM, New York, pp 4274–4285. https://doi.org/10.1145/2858036.2858355
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this entry
Cite this entry
Vatavu, RD. (2023). Gesture-Based Interaction. In: Vanderdonckt, J., Palanque, P., Winckler, M. (eds) Handbook of Human Computer Interaction. Springer, Cham. https://doi.org/10.1007/978-3-319-27648-9_20-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-27648-9_20-1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27648-9
Online ISBN: 978-3-319-27648-9
eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering