1 Introduction

Recent technological advances in the area of virtual reality technologies (VR) have led to a wide spread of use of the VR solutions in entertainment, business, education, rehabilitation and healthcare segments as well as research areas [1, 5]. According to the Gartner Hype Cycle for Emerging Technologies [6], the VR technologies are already emerging as a mature technology, having an important economic impact [13]. There are many reasons for the current success of the VR solutions, ranging from technological ones, e.g. technology of the VR head-mounted display device, user tracking systems fidelity, versatile user input and communication devices, and the increase in the computer processing and graphical power, to the economic affordability of these solutions and general gadget-appreciation in the modern society.

The head-mounted display device (HMD) presents the central part of any VR system. Two high spatial and temporal resolution displays (typically 1080 × 1200 pixels per eye) with 90 to 120 Hz refresh rate are commonly used. This enables an optimal stereoscopic image rendering. In addition, it includes also stereo binaural speakers (speakers, headphones or bone-conductance devices) and precise head/body tracking system. The latter is one of the key components of a good user experience. Namely, the overall low latency of the system means a decrease in time lag between the perceived video information from the user’s eyes and perceived proprioreceptor information from the user’s body. Thus enabling a good user experience. There are several examples of the HMD currently on the market, e.g. the Oculus Rift CV1 [14], HTV Vive [15], Sony PSVR [16] and the OSRV [17]. Mobile solutions, i.e. HMD employing specialised optical lens system with a smart device, such as the Google Cardboard/Daydream VR [18] or Samsung Gear VR [19], represent a more affordable alternative with less fidelity as compared to desktop solutions, but utilising nowadays powerful smart devices. VR mobile solutions represent one of the forms of VR use where high speed and low latency 5G networks could be utilized [7], and can further enhance traditional multimedia and IPTV experience [29]. In both desktop and mobile versions the immersion capabilities of the VR experience is greatly influenced by the graphical realism, where advanced deep learning techniques can be used [32, 33].

Importantly influencing the user experience when using the VR systems is the phenomenon of the VR sickness. Namely, during VR experience users could suffer from symptoms like nausea, headache, and dizziness. Sickness resulting from VR goes by many names including motion sickness, cybersickness, and simulator sickness. These terms are often used interchangeably but are not completely synonymous [24]. Motion sickness refers to symptoms and signs resulting from exposure to real (physical or visual) and/or apparent motion [31]. Cybersickness is defined as a visually induced motion sickness resulting from immersion in a computer-generated virtual environment [30]. The difference between cybersickness and motion sickness in that the user is often physically stationary but has a compelling sense of self-motion because of moving visual imagery. Simulator sickness (SS) is defined as sickness that results from shortcomings of a simulation [26].

Currently there is no generally accepted term that covers all sickness symptoms resulting from VR usage [24]. A more general term is needed that is not restricted by specific causes. In this study we use the term VR sickness when discussing any sickness caused by using VR, irrespective of the specific cause of that sickness. Similar in symptoms to motion sickness, VR sickness can result in nausea, headaches, and dizziness. A comprehensive review of the current state of research methods, theories, and known aspects associated with VR sickness is given in [40].

Current measurements of incidence of VR sickness are questionnaires, monitoring of postural sway and body movement, and physiological state. VR sickness as a perception-based quantity can be observed and measured by different approaches [11, 40].

Simulator Sickness Questionnaire (SSQ) presents a standardized method of measuring the effects of the simulator/motion sickness in various environments, such as aviation simulations, driving and medical rehabilitation [2, 25, 26, 36]. The Disorientation factor (SSQ-D) of the SSQ questionnaire is especially emphasized with the highest impact [9]. A series of studies compared reported higher VR sickness levels in HMD compared with desktop viewing (monitor, projection screen) [28, 35, 43]. The study [22] results indicate that in 3D condition the eyestrain and disorientation symptoms were significantly elevated compared to the 2D display.

Measurement of participant’s behaviour, body movement and autonomous nervous system activity can give another insight into the VR sickness phenomenon. Psychophysiological parameters like heart rate, blood pressure, blood flow, breathing rate and breathing amplitude, body surface temperature, facial thermal map and skin conductance are commonly monitored continuously and non-intrusively [4, 8, 10, 11, 20, 27, 28, 34, 37, 47, 48].

Omnidirectional videos (ODV), also known as panoramic or 360 videos, are an emerging kind of media. ODVs are typically recorded with cameras that cover up to 360° of the recorded scene [39]. VR devices are therefore especially suited for watching the ODV. Effectiveness of ODV and correlation with VR sickness was researched in numerous studies, concluding that i) VR sickness level increases from watching ODV VR HMD as compared to 2D TV [35], ii) VR sickness level is related to physiological arousal of the participant (heart rate, blood pressure, skin conductance, respiration, skin temperature, blood volume pulse, surface EMG) [11, 28, 34, 47, 48], iii) ODVs significantly enhance quality of experience, level of immersion, self-reported sense of presence, intensity of awe etc. [9, 28, 42, 44,45,46].

In this research, we studied whether the level of technology matureness of HMDs affects the perception of VR sickness when watching action video content in 360° 3D video conditions. Five omnidirectional video playback devices were used – a 2D TV set, three generations of the Oculus Rift VR HMD and the mobile VR solution Samsung Gear VR. Perceived VR sickness was estimated using the SSQ questionnaire and measuring physiological parameters (heart rate, respiratory rate, skin temperature and skin conductance). The main contributions - the results of the study indicate differences between HMDs in both SSQ score and changes of physiology. Skin conductance was found to be significantly correlated with VR sickness. This implies the VR technology itself has a significant impact on the users’ VR sickness perception.

2 Methods

The research question was, what is the influence of the selected VR video player device in terms of matureness of the technology (2D vs 3D display, VR HMD display quality, resolution, frame rate) on the VR sickness perception and physiology of the participant. We combined subjective perception reports on VR sickness (by means of the SSQ) with objective measurements (physiological parameters). VR sickness and participants’ physiological response were measured as dependent variables, whereas the independent variables were the features of playback VR devices. The controlled variables included the omnidirectional video clip (ODV), environmental conditions, head-phones, furniture and other auxiliary equipment.

2.1 Participants

We recruited 26 participants, 23 male and 3 female (mean age 24.8 years, SD 5.7 years). The participants were students of mixed background, from technical and social sciences, with average proficiency in Internet, computer and mobile phone service usage. Because participants’ characteristics are associated with the most effective virtual reality experience, i.e. lowest VR sickness), we asked them about their previous experience with computers and gaming. Namely, a PC gamer profile was reported to be more associated with higher predisposition to be immersed and hence experience reduced VR sickness symptoms in the VR environment than a console gamer and a non-gamer [41]. Participants’ colour deficiency was tested using the Ishihara colour perception test [23]. Visually impaired participants used their prescription glasses under the VR HMD.

2.2 Instrumentation

An ODV clip was prepared for the experiment. The action content featured a highly dynamic rollercoaster ride, where the participants were virtually riding the rollercoaster. The video was 110 s long and encoded in high quality (4 k resolution, 60 Mbit/s). The content of the clip was a recording of a roller coaster ride. The roller coaster scenario was selected, because it creates stronger immersion and a higher nausea-level, while the interactive task is more engaging in terms of fun, e.g. more realistic simulation with higher levels of visual flow increases perceived VR sickness [12, 20].

All playback devices played the same video clip. Five different video playback device setups were used (Table 1).

Table 1 Specifications of the five video players used in the study
Full size table

To exclude possible influence of the playback quality itself, the smoothness of the ODV content playback on all five devices had to be ensured. Therefore, we used high performance computers (3x high-end gaming PCs: 7700 K Intel core i7 CPU, 16 GB DDR 4 3000 MHz, Samsung Evo 850 SSD, Nvidia 1070 GPU). Each video playback device was running on a dedicated computer. A GoPro VR player was used for playback. The video on the mobile phone was saved onto the internal flash memory, with similar technical performance as an average computer SSD drive, thus ensuring sufficient performance. To simulate the VR experience on the 2D TV set condition, a VR headset was used as a pointing device. By turning it, the participants could choose their viewport on the TV screen as they desired.

To amplify the user experience of the participant, realistic roller coaster sounds and noises were included in the video clip. To exclude possible influence of audio playback quality, the same model headphones (AKG K141 MK II semi-open studio headphones) were used for all the video playback devices.

During the experiment the physiological parameters were recorded on the non-dominant hand using a multiparameter psychophysiological measuring system (MP150 Biopac Inc.). Measuring modules GSR100C for skin conductance by means of wet Ag-AgCl electrodes on the distal phalanges of the pointer and middle finger, SKT100C for skin temperature by means of a fast thermistor on the little finger, RSP100C for respiratory rate by means of a chest belt, and PPG100C module for heart rate by means of photoplethysmography on the ring finger were used (Fig. 1).

Fig. 1
figure 1

Measuring sites of the physiology measurements (EDA – electrodermal activity, PPG – photoplethysmography, ST – skin temperature, respiration rate was measured using a chest belt)

Full size image

2.3 Task and procedure

The experiment was conducted with one participant at a time. Participant’s task was to assess her/his own perception of the VR sickness after watching an ODV video. Not to influence the perception of the participants, the well-being of the participants was of especial importance. Therefore, the experiment was conducted in a custom designed User Experience Laboratory, simulating living room environment, which presents a typical environment for the VR usage at home. Air temperature, relative humidity and lighting conditions were controlled.

Prior the experiment, participants were informed about the general topic of the study. In the beginning of the experiment, the procedure and aims were briefly explained. After the consent for the experiment was signed, the basic demographic and technology background information was collected. We also collected participants’ sight information (use of spectacles, contact lenses, Ishihara colour perception test), preference of adrenaline sports and experience with playing computer games. Only healthy volunteers without any danger to their wellbeing were selected. The experiment was conducted in a comfortable seating position. Two researchers were present, one interacting with the participant and one operating the equipment and observing the participant.

Each participant watched the action ODV on five different devices successively (Fig. 2).

Fig. 2
figure 2

Procedure of the study. Five HMDs were used in a randomized order

Full size image

The order of the devices was randomized (balanced conditions), e.g. shown in preselected order (Latin Square) [3, 24]. Immediately after watching each clip, the participants filled-in the SSQ questionnaire and estimated their perception of VR sickness. After the experiment, a short follow-up interview was performed and participants’ feedback collected. Throughout the experiment, physiology was acquired. Apart demographic data collecting, the time for initial explanations and filling-in the questionnaires was used for EDA conductive gel to become sufficiently absorbed into the participant’s skin to minimize measuring error of EDA. In the same manner the filling-in of the questionnaires between video clips were used for the physiology baseline levels estimation.

2.4 SSQ questionnaire

As a measure for perceived VR sickness the Simulator Sickness Questionnaire (SSQ) was used [26]. The SSQ identifies 16 symptoms of simulator sickness, and participants are asked to rate them as 0 (none), 1 (slight), 2 (moderate), or 3 (severe). Symptoms on the questionnaire include general discomfort, fatigue, headache, eyestrain, difficulty focusing, increased salivation, sweating, nausea, difficulty concentrating, blurred vision, dizzy eyes open, dizzy eyes closed, vertigo, stomach awareness, and burping. Scores are divided into three subsections of nausea, disorientation, and oculomotor, and total score, with higher scores indicating greater levels of simulator sickness (SS). It has been shown that a total score greater than 20 represents significant simulator sickness, whereas scores less than 5 indicate negligible symptoms [49].

2.5 Data analysis

As an objective measure, user’s physiological response was acquired throughout the experiment. Physiological signals are used in studying psychological state of a participant when dynamics of their changes, i.e. their reactivity, is dependent of the speed and amplitude of signal’s reaction to the task (stimuli). The raw data was analyzed offline and several parameters calculated from it using Acqknowledge 4.1 software (by Biopac Inc.). Additional statistical analysis was performed using Microsoft Excel and IBM SPSS Statistics 22.0. Initially the raw signal was pre-processed (outlier and moving artefacts removed) and afterwards treated by selection of time intervals of interests (duration of video clip). From photoplethysomgraph signal the mean heart rate (HR) was calculated using the built-in Acqknowledge function, and, similarly, from chest displacement signal respiration rate (RR) was calculated. Skin temperature (ST) and skin conductance level and response (SCL and SCR) were calculated within video clip duration. Mean SCL value was selected as a measure for skin conductance’s relatively quick (within seconds) increase and its level during task. SCRs are measure of participant’s momentary arousal and represent pulses in skin conductance signal, which were noted if their amplitude exceeded certain level and the pulse occurred less than 5 s after the beginning of the increase. SCRs were estimated after down-sampling the raw signal to 1 kHz, 0.05 Hz high pass filtering and employing a response threshold of 0.02 microS. Number of SCR pulses was recorded within video clip duration.

Changes in physiology were calculated as the absolute difference of physiological parameters’ level during watching the video clip versus the values from a 120-s baseline period prior this tasks. Increased reactivity of the participant, i.e. increase in SCL, increase in number of SCR pulses, increase of HR and decrease in ST, was expected when the participant was psychologically aroused and lower reactivity if the participant was not particularly interested in the task or even bored.

For every test the threshold of significance p = 0.05 was set. The Shapiro-Wilk Test was used to test for normality. Because of normality violations, differences between groups for each feature (nausea, oculomotor, disorientation, total) were analysed with a Friedman test. To examine the differences between devices in more detail Wilcoxon signed rank test was used. The connection between some interval variables was checked so the Pearson correlation coefficient was used.

3 Results

The participants of the study were 48% active gamers, but only 26% had previous experience with VR devices (GoogleVR, Oculus Rift, Microsoft Hololens). 65% of the participants reported to be inclined to adrenaline sports and action contents. No significant colour deficiency was found using Ishihara colour perception test. One of the research questions was whether preference of adrenaline sports plays an important role in experiencing discomfort while watching videos in virtual reality. We found that the highest negative relationship was with device DK1 (−0.643 < r < −0.461; p < 0.05), which means that participants who were more attracted to adrenaline sports experienced significantly less discomfort with DK1. The same pattern was found in the CV1 case, where participants with a higher preference for adrenaline sports achieved significantly lower scores on the SSQ questionnaires (−0.541 < r < −0.394; p < 0.05). We also verified if the demographic factors (age, gender, vision, colour perception etc.) were related to the user experience of the participants. We have found that the individual’s state of vision (far-sightedness/short-sightedness), colour perception and familiarity with computer games, are factors that were not significantly related to an individual’s user experience in virtual reality. This could be explained with a too uniform and too small sample of participants, as 88.5% of participants were experienced in the computer game playing and 88.5% had no problems with colour perception. Despite the fact that 42.3% of individuals used spectacles, this was not in relation with any measure of user experience.

A minute of typical temporal dependence of physiological signal during baseline period, when a participant was in a relaxed psychological state, is shown in Fig. 3. One can observe the low reactivity of skin conductance level, i.e. very few SCR pulses (marked by * in Figs. 3 and 4) and relatively stable heart and respiration rate, which is typical for calm, relaxed state of the participant. Typical signal during the task, i.e. watching ODV using certain HMD, is shown in Fig. 4. Reactivity of heart rate, respiration rate and electrodermal activity was much higher (number of SCR pulses increased) (Fig. 4).

Fig. 3
figure 3

Typical baseline physiological signals of a participant: SC - skin conductance (in microsiemens), * - SCR pulse, HR - heart rate (in beats per minute), ST - skin temperature (in °C) and RR - respiration rate (in breaths per minute)

Full size image
Fig. 4
figure 4

Typical physiological signals during watching ODV video clips: SC - skin conductance (in microsiemens), * - SCR pulse, HR - heart rate (in beats per minute), ST - skin temperature (in °C) and RR - respiration rate (in breaths per minute)

Full size image

Absolute changes of physiology versus baseline levels for different VR devices are shown in Table 2.

Table 2 Absolute difference of physiological parameters of all participants (n = 26) for different VR players
Full size table

Differences in changes of skin conductance response ∆SCR and heart rate ∆HR indicate that TV decreased the reactivity, i.e. calmed the participants (−2 and − 3, respectively). Changes in heart rate were similar for all players except DK1 where they were in the same order than TV. No additional conclusion could be drawn from changes of skin conductance response (SCR), except that TV, DK1 and SG affected the participants in such a way that they got more relaxed when using them.

Differences in changes of skin conductance level ∆SCL and skin temperature ∆ST with SG device was similar to that of TV (negative sign of ∆SCL indicates that participants were similarly relaxed when using SG HMD as with 2D TV). Watching using other HMDs caused larger changes. In case of mobile HMD (SG) we could speculate that skin conductance level SCL was the only physiological parameter holding certain information on different effects on the physiology of the participant when using different HMD to watch the same video clip (Fig. 5). Figure 5 shows increased physiological reactivity for all HMD except for SG, where it is similar to reference 2D TV.

Fig. 5
figure 5

As an example of data from Table 2 the box plot shows absolute difference of skin conductance level ∆ SCL versus the baseline level when watching ODV clip by means of different VR players (TV, DK1, DK2, CV1 and SG)

Full size image

Changes in respiration rate ∆RR were similar for all HMDs and approximately larger than TVs by 1 beat per minute. In general, respiratory rate and skin temperature changes were insignificant, because most differences between baseline and task periods were obscured by the high variability of both signals.

In Table 3 and Fig. 6 the results of the SSQ of the participants are shown. Descriptive statistics for the SSQ results for all VR players are shown in Table 3. The Shapiro-Wilk test showed that the data did not follow the Gaussian distribution therefore non-parametric statistical tests were used. Friedman and Wilcoxon signed rank tests were used for statistical analysis (Table 4).

Table 3 Average SSQ score for all conditions (M – mean value, SD – standard deviation)
Full size table
Fig. 6
figure 6

Nausea, Oculomotor, Disorientation and Total SSQ score for different VR players

Full size image
Table 4 Differences in VR players (Friedman test, *p < 0.05)
Full size table

In Table 3 the descriptive statistics of individual scales under different conditions is presented. Effect on SSQ score was proven to be significant with TV as compared to other VR players (Fig. 7). The maximum deviation is seen in the case of watching the ODV using TV. There are also differences among other devices, but are not statistically significant (Fig. 7).

Fig. 7
figure 7

Total SSQ score comparison for different VR players

Full size image

Differences on the individual questionnaire scales are statistically significant, which means that the participant’s viewing experience varied depending on the devices through which it was played, i.e. largest difference between all devices while viewing clips via TV over four other headsets, with TV giving lower values and DK2 the highest Total SSQ score (Table 3). The same conclusion could be drawn from Friedman’s test, shown in Table 4. When watching videos by means of different VR players, there were significant differences in experiencing.

The fact that TV caused significantly less VR sickness coincided with physiology measurements in this study (Table 2). We were interested in the conditions under which there are significant differences in experience between different HMDs. Table 4 shows the differences from which it is evident that on most SSQ scales there are statistically significant differences between watching TV and watching using other devices. Differences show that participants experienced significantly less physiological changes when watching the ODV clip with TV than with any other device. This means that TV caused a change in the experience measured by the SSQ questionnaire to a smaller extent than the others. A comparison between the combinations of all conditions, by means of Wilcoxon signed rank test for the SSQ score is shown in Table 5. Significant differences were found in SSQ Disorientation scale between CV1 and DK2, with participants reporting greater disorientation when watching the clip with DK2, which could be backed up with measurements of HR, where DK2 caused higher changes in HR (Table 2).

Table 5 Wilcoxon signed rank test for paired comparison of different VR player on SSQ scale (*p < 0.05)
Full size table

To better estimate user experience when watching ODV using different HMDs we wanted to know the correlation between changes in physiology and perception of VR sickness. Our results indicated a significant correlation of skin conductance level with SSQ scores for mobile HMD player SG. The results showed statistically significant correlation of SCL and level of nausea (r = −0.519, p < 0,05), oculomotor (r = −0.483, p < 0,05), disorientation (r = −0.507, p < 0,05), and total SSQ value (r = 0.533, p < 0,001) when using mobile HMD SG. No significant correlation was determined with any other physiological parameter.

4 Discussion

Our research studied the effects of VR technology on perceived VR sickness and physiology of the participant. In addition to the SSQ questionnaire, the psychological state of the participant was monitored through physiological measures; electrodermal activity (SCL and SCR), skin temperature, respiration rate and heart rate. We were interested in the extent to which changes in physiology correlate with the reported VR sickness, feeling of well-being and annoyance/inconvenience level.

We can conclude that participants enjoying adrenaline sports reported significantly less VR sickness than others when watching the ODV clip using DK1 and CV1 devices.

From the results we could conclude that the technology matureness of the VR playback device (HDM) was important factor for participants’ perception of VR sickness and their physiological response. Similar to other studies [40, 49] the SSQ Total score (in particular SSQ Disorientation) proved to be an important tool for VR Sickness estimation. Our study showed that skin conductance significantly correlated with Oculomotor, Disorientation and Total score of SSQ when watching video using the mobile HMD. Direction was negative, which meant that individuals with significantly higher conductance reported significantly lower changes in experience. An important link was observed on all SSQ scales when watching with SG.

Our results indicated that SSQ Disorientation score might be a suitable and sufficient test for VR sickness and future studies could research if the more comprehensive, Total SSQ score is even needed when estimating VR sickness.

In our study different VR players caused significantly more VR sickness symptoms than a two dimensional TV display, what was expected and proven by both results of the SSQ questionnaire and changes in physiology measurements.

Our results show that CV1 as a technologically most sophisticated VR HMD does not always perform the best in terms of lowest VR sickness effects. Surprisingly, CV1 did only enable second best user experience among the tested devices, although the differences were not significant. On the average the least VR sickness was reported using a DK1 device. We can speculate that the type of the video content (calmer versus action content) might influence the participants’ perception of VR sickness and their physiology. Analysis of the baseline parts of the acquired physiology data showed that participants having higher skin conductance reported lower changes in VR sickness experience. In our study the device DK1 with properties of lower technological level actually induced less VR sickness symptoms than CV1. This can be explained that the technologically superior VR HMD CV1 rendered a more realistic scene (better spatial and temporal resolution, low persistence display), amplifying the VR sickness effects. On the other hand, the older DK1 device could not render all the details, which were lost and the VR sickness effects were less severe. This could further be investigated using other ODV content type.

Therefore, future studies will include new generations of VR player devices and a comparison study of videos of two different video content types (e.g. relaxing, action), i.e. addition of a relaxing neutral video content as a contrast to active content in the present study. In addition, the observation of other physiological parameters, e.g. heart rate variability with SDNN and RMSSD, pupil dilatation and facial EMG is foreseen.

4.1 Limitations of the study

It should be noted that parameters characterizing participant’s autonomic nervous system responses like used in this study, can only serve as reliable indicators of participant’s psychological arousal, and have no reliable correlation with valence. This means that measuring participant’s mental activity is possible, but whether the participant is feeling well or not is a mere speculation. Additionally, effects of measurement anxiety, i.e. anxiety exhibited by changes in physiology of participants who were aware of being observed [21], were neglected.

One of the limitations of this pilot study is the population, i.e. not large enough number of participants, and non-uniform gender distribution of the volunteers in the study.

Practical limitations while performing the experiment included operating HMD, which needed to be careful to enable the participants to adjust the picture optimally. With the mobile SG there was an additional challenge in operation (unpleasant physical position on participant’s head, occasional drifting of the picture etc.) because the device had to be uniformly adjusted across all the participants.

Additionally, the selection of the video clip and its perception could be argued. Our general intention was to use a very distinctive video content, being highly reactive in psychophysiological sense. Following the same principle, sound in parallel to a VR experience was used and the selection of roller-coaster noises and sounds could be argued in the same manner as the video content.

Concerning the accuracy and reliability of physiological measurements there are several possible error sources. The effect of environment needed to be monitored. e.g. air temperature and relative humidity were controlled in order to minimize direct effects on skin temperature and skin conductance and indirect effects on cardiovascular system of the participant. The extraneous acoustical noise level was controlled. The lightning was carefully set to allow comparable conditions for all the participants (light affects photoplethysmographic measurement of HR and influences user experience when watching using 2D TV). Measurement uncertainty of physiology measurement system was checked in a prior study [38] and special attention was given to electrode placement and attachment for avoiding moving artefacts errors.

5 Conclusion

The results of our experiment showed that the level of matureness of HMDs in a VR environment is important for the user’s experience and perception of VR sickness. It significantly affects the SSQ grading and skin conductance, but non-significantly other physiological parameters of the user. VR sickness perception was significantly less affected by TV than by other HMD. The mobile HMD did not induce significantly higher levels of VR sickness as compared to DK2 and CV1, but had negative correlation to changes in skin conductance.

SSQ, in particular the Disorientation part, was proven to be a useful questionnaire for assessing the VR sickness effects.

Preference of adrenaline sports yielded to less expressed VR sickness effects when using CV1 and DK1 devices.