Keywords

1 Introduction

Manipulating objects and expressing gestures is daily found in VR games as well as in the real world. Virtual reality Interaction researches [1] are actively conducted in the field of computer graphics and virtual reality to narrow the gap between the real world and the virtual world. Also, the demand for immersive virtual reality applications is rapidly increasing due to the spread of virtual reality devices such as the Oculus Rift and the HTC Vive. Especially, virtual reality interaction technology, which manipulates virtual objects and expresses gestures in an easy and convenient manner in the immersive environment, is an important factor in virtual reality applications. Our goal is to develop an unified system that allows users to manipulate objects and express hand gestures in a familiar way in virtual reality, just as a user manipulates objects and expresses hand gestures in everyday life.

A considerable number of techniques have been developed to manipulate virtual objects. Ray based selection technique [2] is widely used for object manipulation in the virtual environment that casts a ray from the user’s hand or eye, and selecting the objects colliding with the ray, but this technique involve eye-hand visibility mismatch problem. Argelaguet et al. [3] presented a mapping for ray control, which improve the eye-hand visibility mismatch in cluttered virtual environments. Dam et al. [4] proposed a Kinect based interaction technique that provides convenience to users because it handles user’s movements without an object attached to the human body, but it is difficult to convey a stronger sense of VR immersion to users because haptic feedback and HMD (Head Mounted Display) are not provided. Caputo et al. [5] presented a usability study for object manipulation tasks on a immersive VR setup that is based on the Oculus DK2 for scene display and a Leap Motion controller for gesture interaction, however, this study did not consider how to provide haptic interaction to the user. In this paper, we present a novel virtual reality interaction methods that incorporate object manipulation including haptic feedbacks and hand gestures animation.

In summary, our contributions include: (1) An intuitive interaction method that allows users to easily grab, hold, drop, shake, and haptic feedback on virtual objects. (2) A user-friendly interface that allows the user to press the controller’s touch pad with their fingers to express the desired hand gestures and change the hand gestures.

2 Interaction with Objects

In the virtual reality environment, we manipulate the objects including grab, hold, drop and shake etc. Object manipulation is one of the most important influencing factors of interaction. We design some easy but effective methods to manipulate objects and implement vibration. We also try to make it more realistic to improve the users’ immersive feeling.

2.1 Object Manipulation

To grab the objects, we add the box collider to our virtual hands and the objects which are designed to be manipulated. When user presses the controller trigger button, we check everything that overlapped into the hand collider, then we set the transform of the object to the overlapped hand transform for grabbing the objects. When user holds the trigger button, we keep getting the hand transform and assignment it to the object transform for holding the objects so that the grabbed object can move with hand. When user releases the trigger button, we get the hand position and velocity, then we apply the gravity to the object to drop it. In this way, the dropped object will simulate physics with the position and velocity we get in the last frame before release trigger button as shown in Fig. 1.

Fig. 1.
figure 1

Get the hand position and velocity in the last frame.

Full size image

2.2 Vibration Implementation

Tambourine manipulation is almost same to common objects manipulation except the vibration. When we shake the tambourine, we should have the haptic that the tambourine is shaking. To make the vibration more realistic, we get the velocity of the tambourine in each frame and calculate the acceleration using finite difference method [6]. \( {\text{acc}} = \frac{{\left| {v - v_{p} } \right|}}{{t_{\delta } }} \), \( v \) is the velocity in the current frame. \( v_{p} \) is the velocity in the previous frame. \( t_{\delta } \) is the time between two frames. If the acceleration that we get is bigger than the value1, we make the controller shake. At the same time, when we shake the tambourine intensely, user should also feel the vibration stronger from the controller. We limit the strong level not to be infinitely large. If the acceleration is bigger than the value2, we keep the strongest level without raising as shown in Fig. 2.

Fig. 2.
figure 2

Relationship between vibration and acceleration.

Full size image

3 Hand Gestures

To enrich the virtual reality interaction, we design some hand gestures. User can change his hand gesture by pressing the touchpad in the controllers. In this way, user can get more fun from interaction except object manipulation. In our implementation, we designed 6 hand gestures and simply divide the touchpad to 6 parts corresponding to 6 gestures as shown in Fig. 3. Table 1 shows the method how we divide the touchpad. Then we detect the user’s current touch position and display the corresponding gesture animations. When user releases the touchpad button, we clear all the hand gestures to be ready for the other gestures.

Fig. 3.
figure 3

Correspondence between touchpad region and hand gesture.

Full size image
Table 1. Touchpad division method.
Full size table

4 Results

The proposed VR interaction system were implemented by using Unity3D 2017.3.0f3 and HTC Vive. Figure 4 shows our objects manipulation result. And by recognizing the different tags of objects, we add the vibration property to tambourine. In the result, we can manipulate objects easily and feel the haptic of vibration naturally. Our hand gestures result is shown in Fig. 5 and different gestures can be switched smoothly. We can express our feelings by showing different hand gestures, which is very interesting and effective in virtual reality interaction.

Fig. 4.
figure 4

Microphone manipulation illustration (top). Tambourine manipulation illustration (bottom).

Full size image
Fig. 5.
figure 5

Hand gestures interactions result.

Full size image

5 Conclusion

In this paper, we present a novel virtual reality interaction method for immersive virtual environment including object manipulation, tambourine vibration and hand gestures interaction. Our object manipulation technique includes grabbing, holding, dropping, and shaking. Proposed system provide effective haptic feedbacks corresponding to user’s hand shaking by applying acceleration based on a finite difference method. Also our system generate hand gesture animations in an intuitive manner for users to change the gesture by pressing the touchpad. Our methods allow users interact with the virtual reality easily and naturally. Besides, the hand gestures interaction design is very helpful for users to have a better virtual reality experience.