Abstract
The exisitng conversation system where the user interacts with the virtual system with voice and virtual system replies to the user based on what user speaks. In this context whenever user makes some gestures to communicate with the virtual system, the virtual system will miss out those communications. For example, user instead of speaking, may nod head for “yes” or “no” and user can also use hand signals to respond to the virtual system. If these events are not addressed then the conversation is not very interactive and natural human-like interaction will start losing important information. The paper describes how the user body moments/gestures will help effective conversation with the virtual system and virtual conversation system can understand the user misspelled conversation, missed conversation effectively with user gesture/body movements.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kaushik, Manju, and Rashmi Jain. 2014. Natural user interfaces: Trend in virtual interaction. arXiv preprint arXiv: 1405.0101.
Eshed, Ohn-Bar, and Mohan Manubhai Trivedi. 2014. Hand gesture recognition in real time for automotive interfaces: A multimodal vision-based approach and evaluations. IEEE Transactions on Intelligent Transportation Systems 15 (6): 2368–2377.
Molchanov, Pavlo, Shalini Gupta, Kihwan Kim, and Jan Kautz. 2015. Hand gesture recognition with 3D convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 1–7.
Wu, Erwin, and Hideki Koike. 2018. Real-time human motion forecasting using a RGB camera. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology. ACM.
Badler, I. Norman. 1997. Real-time virtual humans. In Proceedings of the Fifth Pacific Conference on Computer Graphics and Applications. IEEE.
Kuffner, J.James. 1998. Goal-directed navigation for animated characters using real-time path planning and control. International Workshop on Capture Techniques for Virtual Environments, 171–186. Berlin: Springer.
Ng, Kia. 2004. Music via motion: Transdomain mapping of motion and sound for interactive performances. Proceedings of the IEEE 92 (4): 645–655.
Nguyen, H. Katerina. 2001. Method and apparatus for real-time gesture recognition. U.S. Patent No. 6,256,033, 3 July 2001.
Zhou Jie, and Pu Cheng. 2016. System and method for gesture recognition. U.S. Patent No. 9,323,337, 26 Apr 2016.
Huang, Kuang-Man, Ming-Chang Liu, and Liangyin Yu. 2013. System and method for dynamic gesture recognition using geometric classification. U.S. Patent No. 8,620,024, 31 Dec 2013.
Smith Dana, S. 2014. Geometric shape generation using multi-stage gesture recognition. U.S. Patent Application 13/846,469, filed 18 Sept 2014.
Kurakin, Alexey, Zhengyou Zhang, and Zicheng Liu. 2012. A real time system for dynamic hand gesture recognition with a depth sensor. In 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). IEEE.
Chen, L., H. Wei, and J. Ferryman. 2013. A survey of human motion analysis using depth imagery. Pattern Recognition Letters 34 (15): 1995–2006.
Marin, Giulio, Fabio Dominio, and Pietro Zanuttigh. 2014. Hand gesture recognition with leap motion and kinect devices. In 2014 IEEE International Conference on Image Processing (ICIP). IEEE.
Shan, Caifeng, Tieniu Tan, and Yucheng Wei. 2007. Real-time hand tracking using a mean shift embedded particle filter. Pattern Recognition 40 (7): 1958–1970.
Redmon, Joseph, and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv: 1804.02767.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Agnihotram, G., Kumar, R., Naik, P., Yadav, R. (2020). Virtual Conversation with Real-Time Prediction of Body Moments/Gestures on Video Streaming Data. In: Swain, D., Pattnaik, P., Gupta, P. (eds) Machine Learning and Information Processing. Advances in Intelligent Systems and Computing, vol 1101. Springer, Singapore. https://doi.org/10.1007/978-981-15-1884-3_11
Download citation
DOI: https://doi.org/10.1007/978-981-15-1884-3_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1883-6
Online ISBN: 978-981-15-1884-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)