Introduction

The use of visual displays is a crucial part of our daily lives, with smartphones, tablets, TV/PC screens, and virtual reality applications being the most prominent examples of such devices. Unfortunately, the use of these displays can cause unwanted side effects in some users, including nausea, dizziness, fatigue, headache, and eyestrain (Rebenitsch and Owen 2016; Caserman et al. 2021). Visually induced motion sickness (VIMS) is a sensation similar to traditional motion sickness, with the difference that symptoms are not caused by actual, physical movement, but are triggered by stimulation of the visual system (Cha et al. 2020; Keshavarz and Golding 2022). According to a recent online survey with more than 300 participants (Keshavarz et al. 2021), the percentage of users who experienced nausea at least occasionally when using VR technologies or smartphones was approximately 40% and 20%, respectively. Dizziness was even higher for VR usage (45%), and eyestrain was most the most prevalent symptom for smartphone users (70%). These findings demonstrate that experiences with VIMS are highly prevalent in the population.

Theorized causes of VIMS include sensory conflict between the visual, vestibular, and proprioceptive systems (Reason and Brand 1975; Reason 1978; Oman 1990), the inability to maintain postural control (Riccio and Stoffregen 1991), or the contribution of eye movements (Ebenholtz et al. 1994). It has been proposed that VIMS and traditional motion sickness may represent stress responses (Gianaros et al. 2003; Muth 2006) characterized by an increased release of stress hormones (Drummer et al. 1990; Otto et al. 2006; but see Jacob et al. 2023). Although changes in cardiovascular activity (Cowings et al. 1986; Hu et al. 1991), skin temperature (Cobb 1999; Min et al. 2004), breathing (Kim et al. 2005), electrodermal activity (Warwick-Evans et al. 1987; Golding 1992), and gastric activity (Muth et al. 1996; Cheung and Vaitkus 1998) have been noted in various studies, a coherent pattern that allows to objectively quantify and potentially predict VIMS has yet to emerge (Dennison et al. 2016; Keshavarz et al. 2022).

Neurophysiological correlates of VIMS

Our knowledge on the neurophysiological processes involved during VIMS is limited. Over the course of the last decades, insights into the neural basis of general nausea and vomiting have significantly improved (Andrews 2013; Yates et al. 2014), but understanding of the neural correlates of nausea and other symptoms that are specifically related to VIMS or traditional motion sickness remains vague. Studies using electroencephalography (EEG) to identify the neuro-cortical processes underlying VIMS have led to mixed findings (see Chang et al. 2023 for an overview). For instance, Kim et al. (2005) asked their participants to complete a virtual navigation task for approximately 10 min while they recorded EEG data from 9 electrodes (frontal, temporal, central, parietal, occipital) and measured the severity of VIMS after stimulus exposure using the Simulator Sickness Questionnaire (SSQ; Kennedy et al. 1993). Results showed a significant increase in delta power and a significant decrease in relative beta power of the EEG signal at the frontal (F3) and temporal (T3) electrode sites during the navigation task compared to baseline. The changes in the relative delta and beta power were significantly correlated with the level of subjectively reported VIMS measured via the SSQ. That is, participants who reported more severe VIMS showed a stronger increase in relative delta power (F3, T3, O1) and a stronger decrease in relative beta power (F3, T3, T4, O1). Additionally, a significant positive correlation was found between VIMS severity and theta power in the parietal region (P3). The authors speculate that the overall transient EEG slowing might reflect a general stress response. An increase in the delta band in fronto-temporal and central regions during motion sickness has also been reported by various studies using visual stimuli (Hu et al. 1999; Min et al. 2004; Nürnberger et al. 2021) or cross-coupled stimulation to induce motion sickness (Wood et al. 1990; Chelen et al. 1993). Similarly, an increase in theta power over the fronto-central and/or parietal regions was also observed by Wu (1992), Wood et al. (1990), Chelen et al. (1993), and Henry et al. (2022). In contrast, other studies reported a decrease in the alpha band power accompanying the onset of VIMS (Chen et al. 2010; Lin et al. 2013; Naqvi et al. 2015), but an increase in alpha power was also reported (Chuang et al. 2016; Henry et al. 2022).

In another study, Krokos and Varshney (2022) had their participants engage in a VR task using a head mounted display where they were passively moved through a VR scene. Through independent component analysis (ICA) and time–frequency analyses, the authors found strong positive correlations between the subjectively reported level of VIMS and changes in the delta, theta, alpha, and beta band. In contrast, Tauscher et al. (2020) exposed their participants to a moving scene using immersive VR dome projection system while recording EEG data from 16 electrodes distributed over the scalp. The authors then compared the power spectral density of the alpha, beta, gamma, delta, and theta waves between sick and non-sick participants at different points in time (e.g., the moment when VIMS was reported or before stimulus termination). Although the raw data suggested various differences between sick and non-sick participants (e.g., delta and theta waves were more prevalent in the sick group), none of these differences were statistically significant. Overall, these mixed findings demonstrate that the neurophysiological correlates and processes associated with VIMS are not well understood. The aim of the present study was to address this gap in knowledge.

The present study

The main goal of the present study was to explore the neurophysiological correlates of VIMS using EEG measures to deliver additional insights into the neuro-cortical changes associated with VIMS. Given the mixed results found in previous studies, we aimed to further enhance the existing knowledge on how EEG patterns change as a function of VIMS. To achieve this, we exposed stationary participants to a series of moving visual stimuli and recorded their level of VIMS after each trial while recording EEG. We then compared EEG responses for participants who experienced no/minimal VIMS to those who experienced mild/moderate VIMS. Given the mixed findings from previous EEG studies, the current study took an exploratory approach. However, based on previous studies, we were particularly interested in testing whether an increase in power in the slower EEG bands, for instance in the fronto-central and parietal areas, would emerge. We applied time–frequency analysis and event-related spectral analysis to identify EEG activity correlated with VIMS experience.

Methods

Participants

Data were collected from a total of 38 participants. Of those, 5 participants were removed from the data analysis due to poor/incomplete EEG data, resulting in a final sample size of 33 (19 women, 14 men) with an age range from 19 to 44 years (M = 26.70, SD = 6.32 years).Footnote 1 All participants reported being healthy (i.e., no recent history of stroke, traumatic brain injury, vestibular disorders) and having normal or corrected-to-normal visual acuity. The study complied with the American Psychological Association Code of Ethics and was approved by the Research Ethics Boards of the University Health Network and of Toronto Metropolitan University. Participants provided written consent prior to the study and were free to terminate the study at any time without negative consequences. All participants received a $45 gift card of their choice as a token of appreciation for their time commitment.

Stimuli and apparatus

Participants were exposed to a visual stimulus consisting of alternating black-and-white vertical bars that moved to the right or left, inducing the sensation of self-motion (or vection) along the yaw axis. The stimuli were shown on three Lenovo ThinkVision monitors (each 24 inches), arranged at 120° between monitors (see Keshavarz et al. 2017). Display resolution was 1920 × 1200 pixels for each monitor and participants were seated 32 cm from the central screen, resulting in a field-of-view of 228° horizontally and 48° vertically (Fig. 1). Participants were asked to fixate on a red cross presented in the center of the screen during stimulus presentation.

Fig. 1
figure 1

Experimental setup showing the three-monitor arrangement and the visual stimulus. Note that during the experiment, the room was completely dark, with the screens serving as the only light source

Full size image

Stimulus speed was set to either 0.5 cycles per second (slow) or 2 cycles per second (fast), with one cycle defined as the combination of one black bar and one white bar. Participants were exposed to a total of 80 trials, including 40 trials at slow speed and 40 trials at fast speed. However, as the average VIMS scores for the fast (M = 1.59, SD = 1.80) and slow (M = 1.36, SD = 1.59) trials did not differ substantially, we averaged VIMS scores across all trials and did not consider an effect of stimulus speed for the data analysis. Similarly, the direction of stimulus motion (left vs. right) was counterbalanced but not further considered for data analysis (D’Amour et al. 2021). The total of 80 trials was divided into 5 blocks of 16 trials each, with a single trial duration of 40 s (2.5-s acceleration, 30-s constant motion, 2.5-s deceleration, 5-s static). Fast and slow trials were mixed within each blocks, with 8 fast and 8 trials shown per block in randomized order. Short rest breaks were provided between blocks.

Dependent measures

VIMS and vection intensity

The Fast Motion Sickness scale (FMS; Keshavarz and Hecht 2011) is a verbal rating scale ranging from 0 (no sickness at all) to 20 (severe/frank sickness). The FMS was designed to inquire the aspects of VIMS related to nausea, general discomfort, and stomach awareness, but does not capture oculomotor issues or symptoms related to spatial disorientation or dizziness. Various studies (D’Amour et al. 2017; Peck et al. 2020) demonstrated strong correlations between the FMS and the Simulator Sickness Questionnaire (Kennedy et al. 1993), one of the most common measurement tools for VIMS. Vection intensity was measured using a rating scale ranging from 0 (no vection at all) to 10 (very intense vection). Both the FMS and the vection intensity ratings were applied after each trial.

EEG measures

Participants were fitted with a 32-channel EEG headset (ANT Neuro eego™ sports) following the International 10–10 system. The following electrode positions were recorded: Fp1, Fpz, Fp2, AFz, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, POz, O1, Oz, O2 plus the two reference electrodes M1 and M2. The scalp EEG was referenced against CPz and AFz was used as the ground electrode. EOG was recorded using a single electrode attached below to the left eye, in line with the pupil. The EEG was recorded using an eego™ sports (ANT Neuro) amplifier in DC-mode with an online 70Hz low-pass filter and an A/D-rate of 500Hz. The signal impedance was set to below 10kΩ.

Study design and procedure

Participants were post hoc assigned to two different VIMS groups based on average VIMS ratings provided after each trial: participants with minimal VIMS (i.e., average FMS < 1.5) or mild–moderate VIMS (i.e., average FMS score > 1.5). The FMS threshold for dividing the two groups was chosen to separate participants who reported no or only minimal VIMS for the majority of trials from those who reported notable VIMS at least at some point during the experiment. Based on this re-grouping, our sample consisted of 21 participants (13 women) with minimal VIMS and 12 participants (6 women) with mild–moderate VIMS.

A single test session lasting approximately 3 h was conducted. After providing written consent, demographic information (e.g., age, biological sex) was recorded and participants completed a series of baseline measures, including a computerized rod-and-frame test as well as questionnaires on state/trait anxiety, depersonalization, and their susceptibility to VIMS. Note that these baseline measures are not of relevance for the present study and will not be further discussed. Participants were then fitted with the EEG cap and prepared for the presentation of the visual stimulus. Before the beginning of the experiment, the phenomenon of vection was described to the participants using the train illusion (see Kooijman et al. 2022) and two practice trials were presented to familiarize the participants with the experimental protocol and the sensation of vection and VIMS. Participants were then exposed to the visual stimuli separated into 5 blocks of 16 trials each. During each trial, participants were asked to press the left button of a wireless mouse upon perceiving vection and to release it after cessation (see Andrievskaia et al. 2023, for details on the vection onset, vection duration, and EEG results). After each trial, participants had to indicate their level of vection intensity (0–10) and VIMS (0–20) using the buttons of the computer mouse. A short rest break was provided after each block. After completion of the experimental task, participants were debriefed and received their compensation.

EEG data processing

EEG data processing was completed using EEGLab (Delorme and Makeig 2004) for MATLAB (v2021b). A bandpass filter of 1–30 Hz was applied to remove noise. Subsequently, EEG data were down-sampled to 250 Hz (from 500 Hz). Channel information was imported, and the eye electrode was removed from further analyses to avoid any computational problems. Independent Components Analysis (ICA) was then used to identify and automatically reject eye and muscle artifacts (Delorme and Makeig 2004). The continuous EEG data were divided into epochs of 35 s length adjusted to the onset of the motion of the visual input. In other words, each EEG epoch ranged from 0 to 35 s relative to visual motion onset, with the 0–500 ms segment serving as the baseline.Footnote 2 With these settings, the post-baseline EEG within each epoch (i.e., 2.5–32.5 s) included the EEG recorded during constant motion only. EEG epochs containing additional artifacts (threshold limit >  ± 120 mv) were identified and removed (Poonian et al. 2015).

Time–frequency decomposition was conducted using a 3-cycle (0.8-s window length) Morlet-wavelet analysis with a Hanning-tapered window applied. To generate the grand-average scalp topographies to compare the two VIMS groups, the 30 s with constant stimulus speed from the original epochs was extracted and compared (i.e., 2.5–32.5 s post-motion onset). The topography of the EEG activity during this epoch is depicted for five frequency bands (i.e., the delta, theta, lower alpha, higher alpha and beta bands) and averaged for the entire epoch. Group comparisons of the distribution of EEG activity within each EEG band during visual motion presentation between the minimal VIMS vs. the mild–moderate VIMS group were computed by means of permutation tests (2000 permutations). Electrodes that depicted significant differences in EEG activity were selected for further exploration. Based on this, separate event-related spectral perturbations (ERSPs) were generated for each electrode of interest. For the ERSP analyses, the time period analyzed included the acceleration and deceleration of the stimulus, and therefore group differences for each electrode investigated variation across the full 35 s of the stimulus exposure.

Results

Behavioral data

The average FMS and vection scores reported by each of the two VIMS groups are presented in Fig. 2. Non-parametric Wilcoxon tests were calculated to compare the average FMS and vection intensity scores between the two different VIMS groups (minimal VIMS, mild–moderate VIMS). Results showed significantly higher FMS scores in the mild–moderate VIMS group (p < 0.001, r = 0.87) and significantly higher vection intensity ratings in the mild–moderate VIMS group (p = 0.016, r = 0.42). Further, the averaged FMS scores and vection intensity showed a moderately strong positive correlation, Spearman’s r(32) = 0.46, p = 0.007.

Fig. 2
figure 2

Average FMS (left) and vection intensity (right) ratings for each of the two VIMS groups. Error bars indicate SEM

Full size image

EEG results

Figure 3 shows the topographical maps for the power distribution in the EEG signal across the scalp for participants with minimal VIMS and mild–moderate VIMS. Results suggested a difference in delta power between the groups, such that participants experiencing mild-moderate VIMS showed an increase in delta power in the right centro-parietal region (CP2) compared to individuals experiencing minimal VIMS. However, this difference was not significant after correction of the alpha level using the False Discovery Rate (FDR) method. In addition, a difference between the two groups was found in the lower alpha band (see Fig. 3C electrodes Cz); again, after FDR correction, this difference was no longer significant.

Fig. 3
figure 3

Grand average topographical maps, showing mean changes in the A delta, B theta, C low alpha, D high alpha, and E beta frequency bands, 2.5–32.5 s after stimulus onset. The maps on the right indicate regions that significantly differ in frequency band activity within select the time period. Note that p-values shown here are not corrected; after FDR correction, no significant differences showed. The baseline ranges from 0 to 500 ms

Full size image

To further examine potential differences between the minimal VIMS group and the mild-moderate VIMS group, we conducted time–frequency analyses for selected electrodes.Footnote 3 That is, we included electrodes that suggested differences between the two groups in the topographical maps (CP2, Cz) and also added central electrodes (C3, C4) that have linked to changes in motion sickness in previous studies (Hu et al. 1999; Haile et al. 2022). As a result, in the right centro-parietal (CP2) region, greater activity in the lower frequencies (delta and theta bands) showed for the mild-moderate VIMS group compared to the minimal VIMS group, matching the findings from the topographical maps (Fig. 4A). In the midline central region (Cz, Fig. 4B), differences between the two groups appeared to be clustered in the alpha and theta frequency ranges, with participants in the mild-moderate VIMS group showing a decrease in alpha and an increase in theta activity towards the end of the trial. In the other central sites (C3 and C4, Fig. 4C and D, there appeared to be greater activity in the lower frequencies in the moderate VIMS group compared to the minimal VIMS group, particularly in the C4 electrode, although these differences seem rather weak.

Fig. 4
figure 4

ERSPs across the A right centro-parietal (CP2), B midline central (Cz), and C–D central (C3, C4) electrodes during the entire (0 to 35 s) trial time period (for details of epoching the continuous EEG see methods section). The baseline ranges from 0 to 500 ms

Full size image

Discussion

In the present study, we investigated the neural activity associated with VIMS by comparing EEG measures for participants who experienced minimal VIMS at all to those who experienced mild–moderate VIMS during stimulus presentation. Although the differences between the minimal VIMS group and the mild–moderate VIMS group were statistically not significant after FDR correction, our results point towards likely candidates in the EEG signal that may reflect differences between the two groups. That is, an overall increase in delta power in the mild–moderate VIMS group was observed, particularly in the central-parietal region. This finding is comparable to previous work reporting an increase in delta power; however, previous studies reported an increase in delta power specifically in the frontal, temporal, and occipital regions during more severe VIMS, which we did not find (Kim et al. 2005; Chen et al. 2010; Nürnberger et al. 2021). In addition, we also noted a decrease in alpha activity in the central brain regions in the mild–moderate VIMS group, contrasting results from previous studies that found an increase in alpha activity with more severe levels of motion sickness (Chuang et al. 2016; Henry et al. 2022). Although previous work also suggested changes in the beta band (Kim et al. 2005; Krokos and Varshney 2022) associated with increased VIMS severity, we were not able to identify any meaningful differences in this frequency band.

Further investigation of the electrodes at the central and centro-parietal sites solidified the differences found in the topographical maps, indicating an increase in activity across the lower frequencies and a decrease in the alpha band in the mild-moderate VIMS group. Interestingly, the time–frequency analyses at selected recording sites indicated that the development of differences in neural activity in mild–moderate VIMS compared to minimal VIMS is an important factor, because earlier stages of visual motion processing are qualitatively different from later stages. This does not come as a surprise as VIMS symptoms tend to develop gradually with increasing exposure to visual motion. It is worth noting that in our data a qualitative shift is observable between 10 and 15 s of motion perception, suggesting that VIMS may be triggered between 10 and 15 s after stimulus onset. The significance of this temporal signature should be investigated in future studies.

The difference between the two groups in the delta wavelength may have been due to overall greater desynchronization within this frequency range in the individuals reporting minimal VIMS. The variation in the lower (delta and theta) wavelengths seen in participants with more severe VIMS may reflect the processing of a sensory conflict, which is often considered the main cause for VIMS (Reason 1978; Oman 1990). For instance, Nürnberger et al. (2021) found that increasing a sensory conflict between visual and vestibular information resulted in increased power in the lower frequency bands below 10 Hz, suggesting that this shift in power towards delta and theta power may reflect the processing of the sensory conflict. In addition, previous studies investigating the integration of incongruent multisensory information have also found an increase in both delta and theta activity unrelated to VIMS. Delta power was found to increase during a multimodal speech perception task (Pan et al. 2019), or during a Stroop task (Haciahmet et al. 2023) when the sensory information provided were incongruent. Similarly, theta power has been found to increase during incongruent trials of a various Simon tasks (Cohen and Donner 2013; van Driel et al. 2015). Thus, it is possible that changes in delta and theta power reflect the processing of visual–vestibular incongruency. As this congruency is thought to elicit VIMS, it seems plausible that participants who experienced more VIMS also perceived a more pronounced sensory conflict which might be reflected in the increase in delta and theta power. Support for this assumption can also be found in the vection ratings provided by our participants. That is, in the mild–moderate VIMS group, vection ratings were significantly higher compared to the minimal VIMS group, suggesting that this group perceived a stronger conflict between the visual (i.e., suggesting self-motion) and the vestibular and somatosensory (i.e., suggesting stasis) senses.

While similar work has reported an increase in alpha power during periods of reported motion sickness (Chuang et al. 2016; Henry et al. 2022), we found the opposite pattern in our analyses. Alpha power increased later in the trial for participants reporting minimal VIMS compared to individuals experiencing mild–moderate VIMS (however, note that this was not significant when averaged across 30 s of the trial). This may be attributed to the role of alpha band oscillations in sensory suppression. Specifically, alpha band activity has been associated with an inhibition of neural activity linked to task-irrelevant information (Neuper et al. 2005; Foxe and Snyder 2011; Klimesch 2012; Zumer et al. 2014), suggesting that when an individual watches a moving stimulus while stationary, variation in alpha band activity may be measured. Specifically, greater alpha oscillations may be associated with the suppression of contradictory information (in our study design, a lack of supporting vestibular cues that the body is moving) to promote the processing of the moving visual stimulation.

Lastly, across the midline central region, participants with moderate VIMS showed overall greater variability across all frequency bands compared to the minimal VIMS group. This may have been due to the greater range of sickness scores, given that moderate VIMS scores were as high as 11 out of 20, while the highest FMS score in the minimal VIMS group was 5. As a result, it is recommended that future work should consider a narrower definition of moderate motion sickness and identify if that may reduce variability in the EEG responses.

Limitations and future directions

One of the main limitations of the present study is the small sample size. Although our sample size is common in EEG studies (and even exceeds them) (Chen et al. 2010; Henry et al. 2022), the null findings in the different EEG frequencies between the two VIMS groups must be interpreted with caution, as the power to detect large effects was considerably low (1-beta = 0.58).

Additionally, we found a positive correlation between vection and VIMS in the present study. Although vection alone is considered neither sufficient nor necessary to cause VIMS (Lawson and Riecke 2014; Keshavarz et al. 2015), our results suggests that the two are indeed tightly linked to each other. As such, it is possible that the differences in the EEG signal between the two different VIMS groups might represent the different vection ratings in these two groups. Previous studies have in fact reported that vection leads to changes in the EEG signal (see Berti and Keshavarz 2020, for an overview), although these changes typically occurred in the alpha band. To detangle VIMS and vection, future studies should use stimuli that generate vection and VIMS separately. Vection can easily be generated without VIMS (as shown in our minimal VIMS group), but creating stimuli that induced VIMS without vection are more challenging to create.

Furthermore, our study did not create strong sensations of VIMS across all participants, with the average FMS scores being rather low. To maximize the chance to find potential EEG differences, it is recommended to use stimuli that can create stronger VIMS, for instance by applying a different stimulus and setup (e.g., tilting rotating stimulus).

Additionally, in the present study, we compared EEG responses on a group level. That is, we post hoc split our sample into those who, in general, reported minimal VIMS or mild-moderate VIMS and compared their EEG measures across all trials. A different approach for future studies could be to use a within-subjects design comparing trials that elicited minimal VIMS or mild–moderate VIMS. For the present study, we tried to apply this logic; however, only a few participants reported both trials with no VIMS at all and mild–moderate VIMS. Due to the small sample size, we were not able to conduct this analysis in a meaningful way. For future studies, we recommend to design the visual stimuli in a way that they will contain trials that generate no VIMS or moderate VIMS. An alternative approach was applied in studies by Wei et al. (2019, 2023) who compared EEG differences in individuals with a general higher or lower motion sickness susceptibility. This procedure does not require the actual induction of VIMS, which allows more flexibility with regards to the experimental settings (e.g., duration of stimulus, number of trial repetitions).

Finally, it would be interesting for future studies to integrate findings from various imaging techniques such as fMRI (Napadow et al. 2013; Miyazaki et al. 2015) or fNIRS (Zhang et al. 2020). This allows to combine the strengths of each of the different approaches (e.g., high spatial resolution for fMRI, high temporal resolution for EEG) to gain a more complete picture of the neuro-cognitive processes during VIMS.

Conclusion

In the present study, we compared the neural responses associated with VIMS by comparing participants who experienced minimal VIMS or mild–moderate VIMS. Results pointed towards an increase in delta power in participants who experienced mild–moderate VIMS compared to those with minimal VIMS, particularly in the central-parietal region. Interestingly, an increase in alpha synchronization later in the study trial (15 s onwards) was measured in individuals reporting minimal VIMS in the midline central region. No other notable differences in the EEG signal were found between the two groups. Our results suggest that identifying the neural activity associated with VIMS can offer insight in sensory processing underlying or presumably evoking VIMS. Identifying EEG correlates of VIMS may help to distinguish between sick and non-sick states in future studies.