Introduction

Brain–computer interface (BCI) technology provides a way to advance interactions with the outside world for people who are disabled. Steady-state visual evoked potential (SSVEP)-BCI has the advantage of a high information transfer rate (ITR), which has become a focus for researchers in recent years (Maye et al. 2017; Müller-Putz and Pfurtscheller 2007; Lin et al. 2007; Nakanishi et al. 2017; Wang et al. 2016; Lamti et al. 2019). The traditional SSVEP-BCI system uses a series of light-flashing modules with different frequencies as a stimulus, and each stimulus corresponds to an operation command. By recognizing the peak frequency of the electroencephalogram (EEG) induced by the stimulus, it is possible to identify the user’s focused target. Thus, the user’s potential operating intent can be deduced (Chen et al. 2017; Wittevrongel and Hulle 2016). In recent years, information such as phase, color, and motion have been used to improve classification accuracy (Wittevrongel and Hulle 2016; Falzon et al. 2012; Yan et al. 2017; Jin et al. 2012).

The potential change of the cerebral cortex in reaction to the visual stimulus is called the visual evoked potential. The brain responses to SSVEP can be divided into three frequency bands: low frequency band (below 12 Hz), medium frequency band (12–30 Hz), and high frequency band (above 30 Hz) (Herrmann 2001; Wang et al. 2006). The amplitude of SSVEP induced by low frequency and medium frequency stimulation is high, while the amplitude of SSVEP induced by high frequency stimulation is low (Wang et al. 2006). This will limit the available frequency bandwidth and the number of coded targets of SSVEP. Moreover, the EEG induced by SSVEP contains more harmonic components, and the harmonic frequency can not be set as the stimulus frequency of another coding target. This further limits the number of SSVEP-BCI encoded targets. Several attempts have been made to solve this problem. One solution is to use the phase-tagged flashes method (Pan et al. 2011; Lee et al. 2010; Manyakov et al. 2013). The SSVEP flickers were driven by phase-tagged flickering sequences at the same stimulus frequency, and the targets were recognized based on these phase lags. However, the number of phase coded targets is still limited by screen refresh rate when using frame-based ‘on/off’ stimulation method of SSVEP (Chen et al. 2017). Another solution is to use a multi-frequency stimulus method. Each coded target contains two different stimulus frequencies. The focused target can be identified by detecting the two frequencies and their harmonic components. Multi-frequency stimulation methods include sequential multiple frequencies flickering method and simultaneous multi-frequency flickering method. The sequential multiple frequencies flickering method proposed by Zhang et al. (2012) often requires long-term stimulation. The number of coding targets increases with the stimulation time, which greatly reduces the ITR of the BCI system. The simultaneous multi-frequency flickering method proposed by Shyu et al. (2010) placed the spatial positions of the two stimuli sources very close. However, some frequency-peaks might be lost due to the shift of attention (Teng et al. 2010). In recent studies, Chen et al. (2013, 2017) proposed the method of using intermodulation frequencies to increase coding targets. It could increase the number of coded targets at a single flickering frequency by altering color or/and luminance modulated frequencies. Chen et al. used the traditional light-flashing stimulation paradigm. Light-flashing has the advantage of high EEG signal-to-noise ratio (SNR), but long-term use can easily lead to visual fatigue. The parafoveal stimulation can produces less fatiguing stimuli, and is achieved when the stimulus is 2°–5° of visual angle from the foveal center (Waytowich and Krusienski 2015). Waytowich and Krusienski (2015) presented the light-flashing pattern in a circular form, and weakened the light-flashing stimulation by stimulating the parafovea of the human eye. In order to improve the comfort of light-flashing stimulation, we proposed the steady-state motion visual evoked potential (SSMVEP) method to replace light-flashing stimulation with motion stimulation in the previous study (Yan et al. 2018). This new study puts forward a BCI method based on motion and light-flashing hybrid coding to increase the number of coding targets. The light-flashing pattern was presented in a ring form, and the visual angle of the subject’s fovea center and the light-flashing pattern was approximately 3° in this study (please see details in Sect. 2.4). The motion stimulation pattern was placed at the inner circle of the ring. This arrangement not only ensures the comfort of the hybrid coding paradigm, but also ensures that the subject can simultaneously observe the two stimuli without the problem of losing frequency-peaks due to attention shift.

This study used five frequencies to encode nine targets. The motion frequencies were 8 Hz, 9 Hz, 10 Hz, 11 Hz, 9 Hz, 10 Hz, 11 Hz, 10 Hz, and 11 Hz, respectively. The corresponding light-flashing frequencies were 7 Hz, 7 Hz, 7 Hz, 7 Hz, 8 Hz, 8 Hz, 8 Hz, 9 Hz, and 9 Hz, respectively. The main frequency components of EEG induced by the hybrid paradigm were analyzed by power spectrum. The experimental results showed that the hybrid paradigm could stably induce the motion frequency and the light-flashing frequency. Moreover, there was a weak mutual modulation between the two stimuli. Using canonical correlation analysis (CCA)-based target recognition algorithm, the recognition accuracy and ITR of the hybrid paradigm under different time windows were analyzed. The results showed that the hybrid paradigm could be considered for BCI systems. In addition, this study preliminary explored the difference between EEG induced by hybrid stimulation of motion/light-flashing and light-flashing/light-flashing.

Methods

Subjects

Seven males and three females (19–25 years old) were recruited as subjects in this study. The participants were healthy and had normal colour and visual perception.

Experimental equipment

A g.USBamp (g.tec, Austria) was used to collect the EEG. The sampling frequency of the equipment is 1200 Hz. Before the experiment, the reference electrode was placed on the subjects left ear, and the ground electrode Fpz was placed on the forehead. EEGs were also collected from the following 8 channels: O1, Oz, O2, PO3, POz, PO4, PO8 and PO7.

Paradigm design

The same sine formula was used for the motion stimulus and light-flashing stimulus:

$$ M = x + x \cdot \sin \left( {2\pi f_{e} t - \frac{\pi }{2}} \right),L = x^{'} + x^{'} \cdot \sin \left( {2\pi f_{e} t - \frac{\pi }{2}} \right) $$
(1)

where x represents the rotation angle of the motion pattern texture for the motion stimulus, x′ represents the luminance of the ring pattern for the light-flashing stimulus, and fe represents stimulus frequency. The luminance of the ring pattern was changed by changing the RGB of the pattern color. The conversion between RGB and luminance can be calculated by the following formula:

$$ {\text{Luminance}} = {\text{R}} \times 0.299 + {\text{G}} \times 0.587 + {\text{B}} \times 0.114 $$
(2)

In this paper, the initial RGB value of the ring pattern was set to (120, 120, 120), and the variation range of the pattern luminance can be calculated as 0–240 by Eq. (2). The fundamental frequency of the EEG induced by motion stimulus is the flip frequency, which is the frequency by which the direction of the motion changed [Please see motion paradigm in Yan et al. (2018)]. Therefore, if we want to set the stimulus frequency of the motion to 8 Hz, the value of fe should be set to 4 Hz. Since the paradigm pattern was presented frame by frame through the screen, the time t needs to be discretized:

$$ t = \frac{n}{{f_{r} }} $$
(3)

where fr represents the refresh rate of the screen, and n represents the frame number of the paradigm pattern. The hybrid paradigm consisted of a light-flashing pattern and a motion pattern. The light-flashing pattern was presented in an annular form, and the motion pattern was arranged at the inner ring of the light-flashing pattern. The color of the motion stimulus pattern was green and the background color of the display screen was black.

Offline experiment

  1. (a)

    The stimulus paradigms were presented on a screen with a refresh rate of 240 Hz, and the subjects were positioned approximately 60 cm from the screen. The visual angle of the subject’s fovea center and the light-flashing pattern was approximately 3°, and the parafoveal stimulation was achieved for light-flashing stimulus (Waytowich and Krusienski 2015). Five frequencies were used in this study. The motion frequency and light-flashing frequency of each coding target were combined by two of the five frequencies. The layout of the hybrid pattern is shown in Fig. 1. The corresponding motion frequency is f and the light-flashing frequency is F. The offline experiment included nine blocks corresponding to all nine targets (In each block, the subject only looked at one of the nine targets). Each block contained five trials, and each trial lasted 5 s, separated by an interval of 1 s. Between the two blocks, the subjects can rest properly. Nine targets were simultaneously presented on the screen, and numbered 1–9. Before the stimulus begins, one of nine serial numbers appeared below corresponding target indicating the focus target.

    Fig. 1
    figure 1

    The layout position of the hybrid paradigm and its corresponding motion frequency and light-flashing frequency

    Full size image
  2. (b)

    The results of offline experiment A showed that the inter-modulation between the motion frequency and the light-flashing frequency induced by the motion and light-flashing hybrid stimulation was weak (please see details in section offline results). To further study this, this study also analyzed the light-flashing and light-flashing hybrid coding method. We replaced the motion pattern placed at the inner ring with a light-flashing pattern, and the rest of the experimental conditions were the same as those described in section Offline experiment A.

Online experiment

The performances of the proposed motion and light-flashing hybrid coding method were verified on-line. The online experiment included three blocks each containing eighteen trials. We used the randperm function of Matlab to distribute the numbers 1–9 randomly, where two sets of new digital sequences were generated, and then spliced into a sequence containing eighteen numbers. One of eighteen serial numbers appeared below corresponding target indicating the focus target. Each trial lasted 5 s, separated by an interval of 1 s. Eight electrode sites (O1, Oz, O2, PO3, POz, PO4, PO8 and PO7) were chosen to calculate the online accuracy by the recognition algorithm introduced in this study. Each time the serial number of the focused target identified by the recognition algorithm was provided to the subject through the screen. The other components of the design of the online experiment were the same as those of the offline experiment A.

Pre-processing of EEG data

The corresponding EEG data segments were extracted in accordance with the trial start and end times. The Matlab library function detrend was used to remove the linear trends for each channel. Chebyshev band pass filtering of 1–80 Hz was used to remove low frequency drifts and high frequency interferences.

Recognition algorithm

CCA is a nonparametric multivariate statistical analysis method used to determine the linear relationship between two sets of multivariate variables (Horváth and Kokoszka 2012). For two sets of multidimensional variable signals X and Y, the goal of the CCA is to find two sets of linear projection vectors wX and wY, so that the correlation coefficient of the linear combinations wTX X and wTY Y has the maximum value. The maximum canonical correlation coefficient of X and Y can be obtained by maximizing the following:

$$ \rho = \mathop {\hbox{max} }\limits_{{w_{X} ,w_{Y} }} \frac{{E\left( {w_{X}^{T} XY^{T} w_{Y} } \right)}}{{\sqrt {E\left( {w_{X}^{T} XX^{T} w_{X} } \right)E\left( {w_{Y}^{T} YY^{T} w_{Y} } \right)} }} $$
(4)

The multi-channel EEG is denoted as X = [X1 X2 X3, …, XN], where N represents the channel number. The CCA reference signal at the stimulus frequency f0 can be defined as:

$$ Y_{f} = \left[ {\begin{array}{*{20}l} {\cos 2\pi f_{0} t} \hfill \\ {\sin 2\pi f_{0} t} \hfill \\ \ldots \hfill \\ {\cos 2m\pi f_{0} t} \hfill \\ {\sin 2m\pi f_{0} t} \hfill \\ \end{array} } \right],\quad t = 1/f_{s} , \ldots ,l/f_{s} $$
(5)

where m represents the number of EEG harmonics, fs is the sampling rate, and l represents the number of sample points. By calculating the canonical correlation coefficients between X and the reference signals at all stimulus frequencies, and the corresponding target with the maximum correlation coefficient is identified as the focused target.

The hybrid paradigm designed in this study consisted of motion stimulus and light-flashing stimulus, and they had different frequencies. The results of offline data analysis showed that the EEG induced by the hybrid paradigm contained stable fundamental frequency of motion stimulus and its second harmonic, and fundamental frequency of light-flashing stimulus and its harmonic components. The modulation frequency component between the motion stimulus and the light-flashing stimulus was small. Based on the experimental results, this study proposed the following recognition algorithm based on CCA, and the algorithm flow is shown in Fig. 2. Assuming that the motion frequency and light-flashing frequency of the k-th target were fk and Fk. The EEG induced by motion stimulation mainly includes the fundamental frequency of the stimulation frequency and its second harmonic. The EEG induced by the light-flashing stimulation mainly includes the fundamental frequency, the second harmonic and the third harmonic of the stimulation frequency. Thus, the CCA reference signal of the motion stimulus was set to [cos2πfkt sin2πfkt cos4πfkt sin4πfkt], and the CCA reference signal of the light-flashing stimulus was set to [cos2πFkt sin2πFkt cos4πFkt sin4πFkt cos6πFkt sin6πFkt]. CCA was implemented between the collected EEG and the reference signal of motion stimulation, and between the collected EEG and the reference signal of light-flashing stimulation, respectively. Four canonical correlation coefficients can be obtained from the CCA calculation results between the collected EEG and the motion stimulation reference signal, and six canonical correlation coefficients can be obtained from the CCA calculation results between the collected EEG and the light-flashing stimulation reference signal. The first and second largest canonical correlation coefficients of EEG and motion stimulus reference signal, and the first and second largest canonical correlation coefficients of EEG and light-flashing stimulus reference signal were added as the final canonical correlation coefficient of the k-th target. According to the above method, the target corresponding to the largest one is identified as the focused target after calculating the canonical correlation coefficients at all stimulus frequencies.

Fig. 2
figure 2

The recognition algorithm flow

Full size image

Signal-to-noise ratio

The SNR was computed as the ratio between the power spectrum amplitude of a given frequency (fx) and the average amplitude of its six surrounding neighbors (Δf = 0.2 Hz):

$$ SNR(f_{x} ) = a(f_{x} )/\left( {\frac{1}{12}\sum\limits_{l = - 6,l \ne 0}^{l = 6} {a\left( {f_{x} + l \cdot \Delta f} \right)} } \right) $$
(6)

where a(fx) represents the power spectrum amplitude at the frequency fx.

Information transfer rate

The ITR is an important criterion to describe the recognition accuracy and the time required of a BCI system. The ITR was calculated by:

$$ ITR = \frac{60}{T}\left[ {\log_{2} M + \sigma \log_{2} \sigma + (1 - \sigma )\log_{2} \left( {\frac{1 - \sigma }{M - 1}} \right)} \right] $$
(7)

where T is the sum of the stimulation time of each trial and the time interval between two trials, M is the number of targets, and σ is the recognition accuracy.

Results

Offline results

Analysis of offline experimental results of motion and light-flashing hybrid encoding method

For each subject, this study superimposed and averaged five offline experimental data for each target. The obtained data were analyzed by Welch power spectrum. The Gaussian Window was used and the length of the window was 5-s. Onesided was used for power spectral density (PSD) estimate. For SSVEP, the Oz channel usually has the strongest response (Xu et al. 2014). In this study, the Oz channel data was used for power spectrum analysis. We analyzed the offline data of all subjects, and the experimental results of the subject 4 were taken as an example. Figure 3 shows the Welch power spectrum plotted using the OZ channel data of the subject 4, where f represents the motion frequency and F represents the light-flashing frequency. The numbers 1, 2, … marked the frequency with a prominent amplitude on the spectrum. It can be seen from the Fig. 3 that all coding targets induced stable motion frequencies and light-flashing frequencies. For the motion stimulus, the fundamental frequency of the stimulation frequency had a prominent amplitude, and the amplitude at the second harmonic was low. This is consistent with our previous findings that the EEG induced by spiral motion stimulation contains the fundamental frequency of the stimulation frequency and its second harmonic (Yan et al. 2018). For light-flashing stimulus, the amplitude of the fundamental frequency was prominent and the amplitudes at the harmonic components were low. This is related to the intensity of the light-flashing stimulus and is also related to the individual differences between subjects. For the subject 4, the EEG harmonic components induced by the light-flashing were less. For the rest of the subjects, we observed more harmonic components for light-flashing. It can also be seen from the Fig. 3 that the modulation frequency between motion stimulus and light-flashing stimulus was small and the amplitudes of modulation frequency components were low. The modulation frequency F + f existed only in targets 2 and 6. The frequency components F + D × (f − F) (D is the coefficient) existed in targets 3, 4, 6 and 9, but the amplitudes were low. The remaining subjects had the consistent experimental results, that is, there were modulation frequency components F + f or/and F + D×(f − F) in some coding targets, but the amplitudes were low.

Fig. 3
figure 3

The power spectrum of nine coded targets plotted using the Oz channel data for motion and light-flashing hybrid stimulation

Full size image

To further explore the inter-modulation between motion stimulus and light-flashing stimulus, we analyzed the EEG SNR induced by the hybrid paradigm using Oz channel data. As shown in Fig. 4, the EEG SNR of 10 subjects were superimposed and averaged, and the error bars in the figure represent the standard deviation (SD). According to the power spectrum analysis, the modulation frequency induced by the hybrid paradigm may appear at F + f or F + D×(f − F). When the value of F + D×(f − F) is equal to the motion frequency or light-flashing frequency of other targets, the recognition result will be affected. For example, the motion frequency and light-flashing frequency of target 1 are 8 Hz and 7 Hz. It is easy to identify it as another target when the induced EEG contains the modulation frequencies of 9 Hz, 10 Hz or 11 Hz. Therefore, the EEG SNR at 9 Hz, 10 Hz, and 11 Hz were calculated when calculating the EEG SNR induced by the target 1. Similarly, the EEG SNR at 8 Hz, 10 Hz, and 11 Hz were calculated when calculating the EEG SNR induced by the target 2. The remaining coding targets were similar to this. At the same time, the SNR of the following frequencies were calculated: f, 2f, F, 2F, 3F, 4F, and F + f. Where f represents the motion frequency and F represents the light-flashing frequency. Figure 4 shows the fundamental frequency of the motion frequency had the highest SNR, and the SNR of the second harmonic of the motion frequency was low. The fundamental frequency SNR and harmonic components SNR of light-flashing were lower than the fundamental frequency SNR of the motion frequency. This is because the motion pattern was located in the center of the visual field for subjects, which stimulated the fovea of the human eye. While the light-flashing pattern in the annular form stimulated the parafovea of the human eye. At the same time, we can see that there were more harmonic components for EEG induced by light-flashing. Figure 4 shows that the SNR of the modulation frequency were low, and the modulation frequencies SNR for most targets were even lower than the second harmonic SNR of motion frequency and quadruple frequency SNR of light-flashing frequency. Moreover, the SNR of the modulation frequency components for each target had no regularity. The experimental results showed that the interaction between motion stimulus and light-flashing stimulus was weak, and there was almost no modulation effect.

Fig. 4
figure 4

The average EEG SNR of 10 subjects for motion and light-flashing hybrid stimulation

Full size image

Analysis of offline experimental results of light-flashing and light-flashing hybrid encoding method

Figure 5 shows the Welch power spectrum plotted using the Oz channel data of the subject 4. Where fi represents the light-flashing frequency at the inner ring and fo represents the light-flashing frequency at the outer ring. Figure 5 showed there were stable fi, fo and their harmonic components in the spectrum. At the same time, it can be seen that the modulation frequency component between the light-flashing stimulus and the light-flashing stimulus was more, and the amplitude of the modulation frequency was high. For example, the modulation frequency amplitudes at 8 Hz for targets 3, 4, and 8 were high, and the modulation frequency amplitude at 9 Hz for target 6 was high. We superimposed and averaged the SNR of all subjects. The experimental results are shown in Fig. 6, and the error bars represent SD. We analyzed the fundamental frequency, second harmonic, and third harmonic SNR of the inner ring light-flashing stimulus and the outer ring light-flashing stimulus. The SNR analysis method at the other frequencies was the same as in section Analysis of offline experimental results of motion and light-flashing hybrid encoding method. It can be seen from Fig. 6 that the modulation frequency SNR were higher than the third harmonic SNR of the outer ring light-flashing stimulation for targets 2, 3, 4, 6, and 8. However, the SNR of the modulation frequency components for almost all targets in Fig. 4 were lower than the quadruple frequency SNR of the outer ring light-flashing stimulus. The above experimental results showed that the modulation between motion and light-flashing was weak, while the modulation between light-flashing and light-flashing was stronger. According to feedback from the subjects, seven of the ten subjects thought that the stimulation intensity to the human eye was increased after changing the motion pattern located in the inner ring to the light-flashing pattern. Therefore, the hybrid paradigm designed by combining the motion pattern with the circular light-flashing pattern was helpful for improving comfort levels.

Fig. 5
figure 5

The power spectrum of nine coded targets plotted using the Oz channel data for light-flashing and light-flashing hybrid stimulation

Full size image
Fig. 6
figure 6

The average EEG SNR of 10 subjects for light-flashing and light-flashing hybrid stimulation

Full size image

Online results

The stimulation duration of each trial in the online experiment was 5 s. The data of the first 2 s, 2.5 s, 3 s, 3.5 s, 4 s, and 4.5 s of the 5 s data were also analyzed. Figure 7a shows the average recognition accuracy of 10 subjects under different time window lengths, where error bars represent SD. It can be seen that the accuracy increases with the increase of the stimulation duration. Among them, the average recognition accuracy was 92.96% when the stimulation duration was 5 s. This indicated that the proposed SSVEP-based BCI method could be used as a practical BCI system. Figure 7b shows the average ITR of 10 subjects under different time window lengths. It can be seen that the ITR increases first and then decreases as the stimulation time increases. When the stimulation duration was 3 s, the ITR was up to 26.70 bits/min. The increase of stimulus duration can improve the recognition accuracy, but it may also reduce the ITR.

Fig. 7
figure 7

a The average recognition accuracy of 10 subjects under different time window lengths. b The average ITR of 10 subjects under different time window lengths

Full size image

Figure 8 shows the confusion matrix for the stimulation duration of 5 s, 4.5 s, 4 s, 3.5 s, and the color scale denotes the accuracy. Figure 8 shows the target 2 was easily misjudged as target 5, target 3 was easily misjudged as target 6, target 4 was easily misjudged as target 7, and target 8 was easily misjudged as target 6. This is due to there were same motion frequency between targets 2 and 5, targets 3 and 6, targets 4 and 7, and targets 8 and 6. Moreover, the frequency interval of the light-flashing between the two targets was 1 Hz. For example, the light-flashing frequency of target 2 was 7 Hz, and the light-flashing frequency of target 5 was 8 Hz. Although the light-flashing frequencies of the two targets were different, the light-flashing pattern was presented in a ring form and the stimulation was not strong enough. When the frequency interval of the light-flashing was small, it sometimes leads to misjudgment. The motion frequency of target 1 was different from the motion frequency of other eight targets, so the recognition accuracy of target 1 was higher than that of the other eight targets. The above analysis indicates that the setting of the motion frequency and the light-flashing frequency requires a certain optimization method. For example, the frequency interval of motion frequency can be smaller because of the motion pattern was located in the center of the human eye, and the frequency interval of light-flashing frequency can be appropriately larger. In addition, when the two targets have the same motion frequency or light-flashing frequency, the difference of the other frequency should be increased.

Fig. 8
figure 8

The confusion matrix under different time window lengths, and the color scale denotes the accuracy

Full size image

Discussion

The purpose of this study is to propose a BCI method which combines light-flashing stimulus with motion stimulus to increase the number of coding targets. To ensures the comfort of the hybrid paradigm. The light-flashing pattern of the hybrid paradigm was presented in a circular form, and the parafoveal stimulation was achieved. A more comfortable motion pattern was arranged in the inner ring of the light-flashing pattern. This arrangement ensures that the human eye can receive two kinds of stimulation at the same time, which avoids the problem of losing frequency-peaks due to the shift of attention. The method proposed in this paper can encode n(n − 1)/2 targets with n frequencies. In this paper, nine targets were encoded using five frequencies. The motion frequency and the light-flashing frequency of the hybrid paradigm were combined from two of the five frequencies. The offline analysis results showed that the hybrid paradigm could simultaneously induce stable motion frequency and its second harmonic, light-flashing frequency and its harmonic components. All subjects had stable motion frequencies and light-flashing frequencies, and there was no loss of a certain frequency component. The EEG harmonic components induced by the light-flashing stimulation were more, and the EEG harmonic components induced by the motion stimulation was less. For this phenomenon, we still cannot reasonably explain it at present, which deserves further study. The offline experimental results also showed that the mutual modulation between the motion stimulus and the light-flashing stimulus was weak, while the modulation between light-flashing and light-flashing was stronger. This may be due to the different pathways the brain processes for luminance information and motion information.

By analyzing the recognition accuracy and ITR of the hybrid paradigm, we can see that the accuracy increases with the increase of stimulus duration, and the ITR increases first and then decreases with the increase of stimulus duration. Increasing stimulus duration is beneficial to improve recognition accuracy, but it will reduce ITR and cause visual fatigue of subjects. By analyzing the confusion matrix of recognition accuracy, it can be seen that when the two targets had the same motion frequency and the light-flashing frequency was very close, misjudgment will easily occur. The light-flashing pattern was presented in a circular form, and it mainly stimulated the parafovea of the human eye. Therefore, its stimulation intensity was weak. In this way, two targets with close light-flashing frequencies were difficult to distinguish. In subsequent design, we need to optimize the setting of the motion frequency and the light-flashing frequency of the hybrid paradigm. The motion pattern was located in the center of the human eye, and the frequency interval between the two targets can be smaller. For the target with small motion frequency interval or the same motion frequency, it is necessary to increase the interval of light-flashing frequency as much as possible.

Many studies have been conducted in the field of BCI multi-target coding based on SSVEP. It should be noted that it is difficult to make a direct comparison of these studies due to differences in stimulation paradigms, signal processing and experimental settings. The color-luminance modulation method proposed by Chen et al. achieved an accuracy of 93.83% and ITR of 33.8 bits/min (Chen et al. 2013), and obtained better performances than the method proposed in this paper. However, this method adopted the light-flashing stimulation paradigm, which is less comfortable and easily triggers photoepileptic seizures (Fisher et al. 2005). The coding method proposed in this paper used the motion stimulation and ensured the comfort of the paradigm. The simultaneous multi-frequency flickering method proposed by Shyu et al. found that some frequencies might be lost in the detection process (Shyu et al. 2010). The experimental results showed that the method proposed in this paper could induce stable motion frequency and light-flashing frequency without the problem of missing frequency components. The sequential multiple frequencies flickering method proposed by Zhang et al. (2012) achieved an accuracy of 93.75% and ITR of 24.28 bits/min. However, this method often requires long-term stimulation. The proposed method in this paper obtained a higher ITR than the method proposed by Zhang et al. The comparisons with the previous studies indicated that the proposed method can be considered for BCI system.

Conclusion

This study combined the cyclic light-flashing stimulus with the motion stimulus, and designed a hybrid paradigm to increase the BCI coded targets. With this method, we coded nine targets using only five frequencies. The average accuracy was 92.96%, and the ITR was 26.10 bits/min. The experimental results also showed that the interaction between motion stimulation and light-flashing stimulation was weak.