Keywords

1 Introduction

The development of the personal computer (PC) and the Internet introduced the new behavior of “online shopping” since 1980. The first online shopping behavior—TV shopping, transformed the traditional shopping in 1984 [1]. Traditional shopping allows people to physically interact with the products through a multitude of senses: sight (ophthalmoception), hearing (audioception), taste (gustaoception), smell (olfacoception or olfacception), and touch (tactioception) before making decisions. In contrast, during online shopping, people are restricted by visual (monitor) and auditory (speaker) inputs and can only use the “browser to search keywords” or use the “classification” to find the products of interest. Nevertheless, internet endows the power of easy and quick browsing/comparing the items/stores across the world, which is not achievable in the physical world.

However, there are many limitations in current 2D online shopping— the shopping experience, shopping environment, interface, or even the restricted screen size. The challenge of this research lies in how to create a more natural and intuitive shopping environment by using the benefits of physical shopping and 2D online shopping.

VR devices have recently become more popular, less expensive and widely integrated into multiple fields [2,3,4,5]. Thus, VR could potentially be used as an alternative bridge to connect the physical and online shopping stores. Since this is an ongoing project and we have previously discussed the preliminary scenario of the VR online shopping [6], we will implement a system prototype—“VROS” by using Samsung HMD Odyssey+ as the VR hardware and Unity as the platform base. To evaluate the possibility of VROS, we used protocol analysis to test the system by two-stage experiments.

2 Methodology and Steps

To create a more natural and more intuitive shopping experience, we combined both the significance of “online shopping” and “physical store shopping” into the virtual reality online shopping (VROS) system by using VR hardware. Through the VROS system, we expected the users can smoothly, safely, appropriately, and easily shop at the VR, and even better than the physical experience. The methodology and steps are as follow.

2.1 The First Step: System Concept

The concept of the VROS was established by the results of the online shopping preferences from the view of “environment”, “product” and “experience” [6].

2.2 The Second Step: Implementation

In order to implement the VROS prototype, the step included three stage:

  • 3D modeling. We built the 3D model as the VROS environment based on the result of the first step.

  • System framework. We discussed the system framework, system installation including the selection of hardware and Unity platform establishment.

  • Manipulation. We discussed the manipulation in VROS.

2.3 The Third Step: Scenario Demonstration and Evaluation

In order to create an adequate VR online shopping environment, we used the scenario demonstration and two-stage experiments to evaluate the VROS system as described below:

  • Experiment S1. Shopping experiments in “traditional 2D online shopping environment”.

  • Experiment S2. Shopping experiments in “VROS environment”.

3 System Concept

The VROS concept was followed by the previous study of 2D online shopping experience [6]. The VROS system consists of three main structures: shopping environment, shopping product and shopping experience (see Table 1).

On the other side, shopping happens in our daily life. All the shopping goods we purchased physically and virtually exist in our living environment. “Home” is the most fundamental space for human beings. Therefore, we used “home-hub” as the core concept of the VROS. All the shopping activities would start from the virtual home-hub, which could teleport to the worldwide e-shops. Through VROS, whether the “items are already purchased” or the “items are on the wish-list”, the users can easily arrange them into the home-like environment. There are five characters as below.

Table 1. Comparison between physical shopping, 2D online shopping and VROS.
Full size table

3.1 Home-Hub

The VROS was an entrance of online shopping home spot. The layout can be customized by the users. In home-hub, there are four teleports in four different space (kitchen, living room, home library and walk-in closets), which were represented as different entrances of shopping categories (Fig. 1). For instance, the kitchen can be teleported to grocery, such as real farm or “Coop”; the living room can be teleported to the entertainment, electronics or home furniture, such as IKEA; the home library can be connected to “Amazon”; and walk-in closets on 2nd floor can be teleported to fashion shop, such as Uniqlo or Macys.

Fig. 1.
figure 1

Concept of home-hub.

Full size image

3.2 Hand Gesture Control

In the VR environment, the user is able to use the hand gesture to naturally interact with the virtual products to enhance the ability of physical shopping experience, such as grabbing or touching a product, 3D panorama appreciating the products or putting the products into the bag (see Fig. 2).

Fig. 2.
figure 2

Hand gesture in VROS (grab, hold, flip, throw and drop).

Full size image

3.3 Product Display and Wish-List

In the VROS home-hub, the product display would be the same as in the physical shop. The products on the wishlist were shown in grayscale (see Fig. 6-H round neck sweatshirt), but can be arranged in VROS home. When the user picked up the product (such as Fig. 6-C wooden rocking horse) the product would turn back to the original (wooden color).

3.4 Floating Interface (Information and Shopping List)

When the user stopped by or got close to the products, the floating interface emerged with the product information and rating. In order to quickly check the shopping bag items, the shopping bag lists were shown on the wall of the living room.

3.5 Collaboration Shopping

Shopping is usually a very personal behavior. However, according to the previous study [6], most people might share shopping lists with specific people (such as family, colleagues or friends). In the VROS, users could customize a specific person as a collaborative shopping friend. When your collaborative shopping friend was online, the grayscale person would turn into color. At the same time, the user can invite him/her to join the shopping.

4 System Implementation

4.1 3D Modeling

In order to implement a “home-hub” based VROS prototype, first, we created a 3D model of VROS shopping environment. The VROS was composed of five main spaces: living room, kitchen, home library, walk-in closet, and farm. Different spaces contained specific products. And each space will be redirected to the related online shopping stores (see Table 2).

Table 2. 3D modeling and elements.
Full size table

Since the VROS was performed on the Unity engine, it is difficult to build a complicated model. Hence, the 3D model (including scene, space, and elements) was established in Rhino. The final model would export as .3Ds file and import to the Unity.

4.2 System Framework

In this research, we used Unity 2018.3.9 as our virtual space environment and “Samsung HMD Odyssey+” as our hardware, which included one headset and two motion controllers. To implement the VROS, the system framework was introduced as shown below (Fig. 3). The user was able to control the VR world by wearing headset (Samsung HMD Odyssey+) and holding the pair of motion controllers. The headset was embedded with a head tracker, which monitored the user’s head location. And the user holding the motion controllers are able to input the signal of user’s actions (such as moving or grabbing objects) to the computer. Meanwhile, the user would get the feedback from the interactive scene (projecting by VR engine: Unity) in real time through the VR system.

Fig. 3.
figure 3

System framework.

Full size image

To install the VROS system, there are three parts as below.

  1. 1.

    Setup headset: since the Samsung HMD Odyssey+ was compatible with Windows Mixed Reality, in Unity, there are only two steps to finish setup. First, change the environment to Universal Windows Platform and choose “XR Settings/Virtual Reality supported”. Second, import “Windows Mixed Reality Toolkit” to assets from Unity.

  2. 2.

    Import 3D model: we build a 3D model (4.1) in Rhino and import 3ds file to Unity.

  3. 3.

    Combination of headset, controllers and Unity: in Unity, we create a camera as the view of VR headset. To display the virtual controller in VR, we imported “motion controller” from assets, which the users could be more immersive in VR environment when seeing the handed controllers.

4.3 Manipulation

Move and Teleport.

In order to experience physical, simple and quick shopping (like 2D online shopping), the VROS used the point to point quick moving to other spaces instead of walking in the VR environment. The user can naturally move forward/backward by thumbing up/down the motion controller. Compared to traditional VR environment, to teleport to a specific product, the user can easily point the “blue spot” near the product by using the motion controller instead of using the keyboard: A, S, D, or W (see Fig. 4).

Fig. 4.
figure 4

Point to point navigating.

Full size image

Interaction with Product.

In order to simulate grabbing an object in reality, the user can intuitively pick up or grab a product by holding the side button (Fig. 5). The product will drop off if the button is released. Also the user can transfer an object from one hand to the other.

Fig. 5.
figure 5

Motion controller.

Full size image

Product Display and Scale.

In physical shopping, the products would be placed onto the shelves or showroom. However, in 2D online shopping, the user can only imagine the real products by watching the small pictures at the restricted 2D screen (e.g. magnifying larger or smaller the pictures of the product). As to Table 3, shopping in both “physical store” or “VROS”, the user can naturally and easily experience the original scale of the products, even trying on the clothes in VROS. In the future version, to increase the shopping efficiency, we will add the feature: “changing style”, “color” or “size” into the functions.

Table 3. Evaluation experiments.
Full size table

5 Scenario Demonstration and Evaluation

5.1 Scenario Demonstration

Jane wanted to do some shopping. She put on the headset and calibrated the relation of headset and controllers. And then she started the VROS system and went into the home-hub. The initial spot started at the living room (Fig. 6-A). First, she saw the wooden rocking horse in grayscale that she would like to buy for her daughter, was on the floor (Fig. 6-C). The wooden horse turned to color when she picked it up and put it into the shopping bag. Meanwhile, she found her father—“John” just came online and turned into color from grayscale. John asked Jane to prepare Heineken beers for his visit tonight. Jane then put Heineken beers into shopping bag (Fig. 6-B). Interested in a newly released book, she used motion controllers to move to the second floor. When she picked up the book from the bookshelves, the information of the books—“How Architecture works” appeared next to the virtual book (Fig. 6-D, E, F). Since she liked it a lot, she naturally put it into the shopping bag. Then she went to the walk-in closets. She found a beautiful round neck sweatshirt (Fig. 6-G). The sweatshirt turned to pink from grayscale when she took it out from the closet, and the details (material, price and rate) of the shirt appeared next to it (Fig. 6-H, I). To prepare dinner, Jane realized she need to get some fresh food at the grocery. She went downstairs to the kitchen and checked the memo from the refrigerator (Fig. 6-J, K). At the same time, she was teleported to the “farm”, her favorite grocery shop (Fig. 6-L, M, N). She finally bought brown eggs, lamb ribs and a bottle of milk. And she went back to home and check the shopping bag from the wall of living room (Fig. 6-O).

Fig. 6.
figure 6

System demonstration: A) Jane started the VROS and enter the home-hub; B) She picked up a bottle of Heineken from the table; C) She saw the wooden rocking horse and she put it into the wishlist; D) She went to bookshelves on second floor; E) She picked up a book “How Architecture works” from the bookshelf; F) She picked up the book and put it into shopping bag; G) She went to the walk-in closet; H) She found a round neck sweatshirt; I) It turned to original color “pink”, when she hold the cloth; J, K) She went downstairs to the kitchen, she found the reminder from the refrigerator; L, M, N) She went to the farm to buy eggs, lamb ribs and milk; O) She went back to the home-hub to check the shopping bag from the wall.

Full size image

5.2 Evaluation

In order to evaluate and compare the differences between the VROS prototype and the 2D online shopping, we created evaluation experiments to test the system. We recruited two subjects (S1 and S2, between 25 and 40 years of age), who prefer going online shopping (including worldwide e-shops) than physical stores. To test the VROS, each subject was individually requested to do two-stage experiments: Part 1. Shop in 2D online shopping stores; Part 2. Shop in the VROS (see Table 4). In each experiment, the subject was assigned to shop the same 9 items: Heineken beer, eggs, milk, lamb rib, books, clothes, Kate bag and wooden horse. The goal of the experiments is to evaluate the differences when the users shop the same goods in different platforms. As to the experiments, we used “audio/video retrospection” and “coding system”, as part of protocol analysis [7, 8], which transformed verbal and video into protocol and reveal thinking process to verify the possibility of the VROS.

Table 4. Evaluation experiments
Full size table

During the two stage experiments (see Table 4), the subjects were requested to think aloud and recorded by video and audio recorders. The audio/video retrospection would be transferred into text, split into segments and then marked by defined coding scheme (see Table 5).

Table 5. Coding scheme
Full size table

The results of two-stage experiments showed that S1 spent approximately similar amount of time shopping in both the VROS (Exp S1-1) (see Fig. 7) and the 2D online shops (Exp S1-2) (see Fig. 8), even though this was his/her first time using the VROS platform. S2 spent only 2 min longer in the VROS. It seems that, in the VROS, both S1 and S2 can easily and simply finished the tasks without any doubts. Although in the current version of VROS, every product has only one option and the subject can simply buy the goods without spending time to make the decision. However, in 2D online shopping, to find the specific products, the coding scheme (Exp S1-2) (see Fig. 7) showed that S1 kept using “GS”, “ES”, “W”, “R” and “B” to make the decision.

Fig. 7.
figure 7

Exp S1-1(VROS) coding scheme

Full size image
Fig. 8.
figure 8

Exp S1-2 (2D online shopping) coding scheme

Full size image

Compared to the coding scheme in Exp S1-1 (VROS) and Exp S1-2 (2D online shopping), the products in home-hub based VROS has been “categorized” and adequately placed in related home-hub “space”. For instance, grocery can be found in refrigerator connected to the farm or supermarket. The coding scheme “M and N in VROS” is approximately equal to “GS, ES and R in 2D online shopping”. The results indicated the home-hub (space classification), in which users could panorama viewing the products, augmented the shopping experience in virtual space rather than repeatedly searching on the multiple webpages (see coding scheme: ES and GS) in the restricted 2D screen.

The results of Exp S2-2 are similar, S2 spent most of time checking the products over and over across multiple website while buying the Kate bag. He/she was debating between the “official website” and “second-hand shop (like eBay)”. Further VROS features will be improved by adding the similar or different condition products in the same space.

6 Conclusion

The objective of the research is to explore a new vision of online shopping platform by using the VR technology (Virtual-Reality Online shopping system/VROS). VROS incorporates the concept of “home-hub”, which could be not only a product storage or showroom but also a “teleport” connecting to different online shopping stores, such as Amazon, Uniqlo or Coop…etc. Either products you wish to buy or you already have could be rearranged into the home-hub. Five different spaces (living room, kitchen, home library, walk-in closet and farm) are created in the VROS. The users can shop intuitively and naturally from specific home-hub space (such as the walk-in closet) classification and teleport to the related online shopping stores. It breaks the limitation of using keywords or text classifications to find the products.

By using protocol analysis, the results show the possibility of VROS in different views. The users can easily, quickly and adaptively shop in VROS without training. Although the VROS is an earlier system prototype and has not gone online yet, the VROS poses a new experience of online shopping. The feedbacks from the subjects implied that products arranged in the personal virtual space would encourage the users to think carefully before purchasing. From the point of the sellers, it would be good if the advertisement can be ubiquitous embedded into every personal virtual home, which might also be helpful to enhance product sales volume.

The VROS is a system prototype and needs continuous improvement in the future. When buying daily goods, the subjects preferred to use the traditional 2D online shopping rather than the VROS. This is because that they can quickly find the products they need. We believe that the VROS can be widely used in the future if more people get used to this new means of shopping behavior. One other feedback from the subjects is that “point to point teleport” sometimes is not easy to control. In the future, the VROS will be improved by adding the moving option of “walk.” Compared to the 2D online shopping stores, the users have multiple option to find the goods before making a decision. Hence the future VROS will include similar or different condition products or even the different e-commerce options.

In the future study, the VROS can approach “VROS world” by connecting to the social network (e.g. Facebook). By extending the 2D online shopping stores to the 3D online shopping stores, every user will be represented as avatar and hold a customized house and experience physical shopping atmosphere without being restricted into a physical space, and can quickly switch among the places to shop around the world (see Fig. 9).

Fig. 9.
figure 9

Concept of VR world

Full size image