Abstract
In the environment of mixed reality (MR) or augmented reality (AR), there have been several previous works dealing with user interfaces for manipulating and interacting with virtual objects aimed at improving immersive feeling and natural interaction. However, it is still considered that there must be much progress in supporting human behavior-like interactions for providing control efficiency and natural feeling in MR/AR environments. This paper proposes a tangible interaction method by combining the advantages of soft interactions such as hand gesture and MR and hard interactions such as vibro-tactile feedback. One of the main goals is to provide more natural interaction interfaces similar to the manipulation task in the real world by utilizing hand gesture-based tangible interactions. It also provides multimodal interfaces by adopting the vibro-tactile feedback and tangible interaction for the virtual object manipulation. Thus, it can make users get more immersive and natural feeling in the manipulation and interaction with virtual objects. Furthermore, it provides an alternative instruction guideline based on the analysis of the previous interaction while manipulating virtual objects, which makes it possible for the user to minimize manipulation errors during the interaction phase and the learning process, which guides the user to the right direction. We will show the effectiveness and advantage of the proposed approach by demonstrating several implementation results.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Weiser M (1999) The computer of the 21st century. ACM SIGMOBILE Mobile Computing and Communications Review 3:3–11
Lee JY, Rhee GW, Kim H, Lee K-W, Suh Y-H, Kim K (2006) Convergence of context-awareness and augmented reality for ubiquitous services and immersive interactions. LNCS 3983:466–474
Schmalstieg D, Reitmayr G (2005) The world as a user interface: augmented reality for ubiquitous computing. Proc. of Central European Multimedia and Virtual Reality Conf., Prague, Czech Republic
Piekarski W (2004) Interactive 3D modeling in outdoor augmented reality worlds. PhD Thesis, The University of South Australia
Ong SK, Yuan ML, Nee AYC (2008) Augmented reality applications in manufacturing: a survey. Int J Prod Res 46:2707–2742
Seichter H (2007) Augmented reality and tangible user interfaces in collaborative urban design. Proc. of the 12th International CAAD Futures Conf., Sydney, Australia
Sidharta R, Oliver J, Sannier A (2006) Augmented reality tangible interface for distributed design review. Proc. of the International Conference on Computer Graphics, Imaging and Visualization 464–470, Sydney, Australia
Lee JY, Rhee GW (2008) Context aware adaptable ubiquitous car services using augmented reality. Int Jo Advan Manuf Technol 37:431–442
Lee JY, Seo DW, Rhee GW (2008) Visualization and interaction of pervasive services using context-aware augmented reality. Expert Syst Appl 35:1873–1882
Kato H, Billinghurst M, Poupyrev I, Imamoto K, Tachibana K (2000) Virtual object manipulation on a table-top AR environment. Proc. of ISAR 2000 111-119, Munich, Germany
Park H, Moon H-C, Lee JY (2009) Tangible augmented prototyping of digital handheld products. Comput Ind 60:114–125
Lee JY, Rhee G, Park H (2009) AR/RP-based tangible interactions for collaborative design evaluation of digital products. Int J AdvanManuf Technol 45:649–665
Terry M, Cheung J, Lee J, Park T, Williams N (2007) Jump: a system for interactive, tangible queries of paper. Proc. of Graphics Interface 2007 127–134, Montreal, Canada
Hong D, Looser J, Seichter H, Billinghurst M, Woo W (2008) A sensor-based interaction for ubiquitous virtual reality systems. Proc. of International Symp. on Ubiquitous Virtual Reality 2008 10–13, Gwangju, South Korea
Buchmann V, Violich S, Billinghurst M, Cockburn A (2004) FingARtips: gesture based direct manipulation in augmented reality. Proc. of the 2nd International Conf. on Computer Graphics and Interactive Techniques in Australasia and South East Asia 212–221, Singapore
Ott R, Thalmann D, Vexo F (2007) Haptic feedback in mixed-reality environment. Vis Comput 23:843–849
White M, Petridis P, Liarokapis F, Plecinckx D (2007) Multimodal mixed reality interfaces for visualizing digital heritage. Int J Architect Comput 5:322–337
Thomas BH, Piekarski W (2002) Glove based user interaction techniques for augmented reality in an outdoor environment. Virtual Real 6:167–180
Robert W, Linderman JN, Sibert JL, Cutler JR (2002) Handling of virtual contact in immersive virtual environments: beyond visuals. Virtual Real 6:130–139
Veigl S, Kaltenbach A, Ledermann F, Reitmayr G, Schmalstieg D (2002) Two-handed direct interaction with ARToolkit. Proc. of IEEE First International Augmented Reality Toolkit Workshop, Darmstadt, Germany
Yi X, Qin S, Kang J (2009) Generating 3D architectural models based on hand motion and gesture. Comput Ind 60:677–685
ARToolkit (2005) http://artoolkit.sourceforge.net
Bowman DA, Wingrave CA (2001) Design and evaluation of menu systems for immersive virtual environments. Proc. of IEEE Virtual Reality 2001 149-156, Yokohama, Japan
David H (1987) Statecharts: a visual formalism for complex systems. Sci Comput Programming archive 8:231–274
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lee, J.Y., Rhee, G.W. & Seo, D.W. Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. Int J Adv Manuf Technol 51, 1069–1082 (2010). https://doi.org/10.1007/s00170-010-2671-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00170-010-2671-x