Keywords

1 Introduction

Steady state visually evoked potential is a strategy that increase the energy in brain activity after the presentation of a visual stimulus modulated at a fixed frequency that can be measure in EEG signal as a magnitude rise at the stimulation frequency. The stimulus is presented as message that a subject must select from a group of stimuli with other messages. This stimuli selection has made SSVEP a successful processing strategy in spellers [1,2,3], games [4], rehabilitation with robot [5], prosthesis control [6] or any application for communication and control purposes.

The research to develop new processing methods for SSVEP has been generated a wide variety of algorithms reported in literature for feature extraction and classification. The most common models for feature extraction are Discrete Fourier Transform (DFT) [7, 8], Canonic Correlation Analysis (CCA) [9,10,11,12,13], Spectral Power Density (SPD) [14], Discrete Wavelet transform (DWT) [15] and Principal Component Analysis (PCA) [16, 17]. For classification, the most used methods are based on Artificial Neural Networks (ANN) [18,19,20] and Support Vector Machine [4, 15, 21].

However, most of the SSVEP methods reported in literature needs many EEG input channels and are computationally complex. Consequently, the Brain Computer Interfaces (BCI) that acquire and process SSVEP signal, require computer systems with high processing capabilities and these BCI may cause discomfort to the users due to electrodes positioned in the scalp. Hence, in order to develop a ubiquitous BCI embedded systems, this paper proposes a low computational cost method for SSVEP processing (EP-SSVEP) that use a single EEG channel and is based on Canonical Correlation Analysis and a Feedforward Neural Network (FFNN). To the experiments, we also propose a ubiquitous and comfortable-to-use BCI embedded system with one EEG channel, a portable acquisition device and a small size embedded processor.

The rest of the paper is organized as follows: Sect. 2 presents the BCI embedded system and the dataset acquired for experiments. Section 3 reports the graphical interface that generates the stimuli to evoke the SSVEP. Section 4 describes the proposed method EP-SSVEP. Section 5 presents the results and finally, Sect. 6 reports the conclusions.

2 BCI Embedded System and Dataset

The design of BCI embedded system for experiments was based on three aspects: acquisition device, number of electrodes and method for SSVEP processing.

The acquisition device for the system is the Cyton board of OpenBCI® and it was selected because of its low cost, portability and its SDK is compatible with commercial embedded boards.

The criterion for the number of electrodes is based on the fact that as the number of channels to acquire EEG signal decreases, the comfort of the system increases, and the complexity of the algorithms falls significantly. Then, we develop an experiment to find the best EEG channels to acquire the SSVEP frequency. In this experiment, we read the EEG signals from eight subjects using electrodes connected to the EEG channels O1, O2, Oz, the ground A1 and the reference Fpz, which are standard positions of 10–20 international system. These channels were selected because according to Lee et al. [22], the occipital lobule is the best part of the brain to detect the SSVEP frequencies when a subject stares a stimulus. Results of our experiments show that the EEG channel Oz generates an SSVEP with the necessary properties to design a system that detects the stimulus that the subject stares.

Thus, the proposed method was designed considering a single EEG channel as input and the method must allow online applications in commercial embedded boards.

These aspects were considered in the design of the BCI embedded system whose components are showed in Fig. 1. The system is composed of a visual interface to evoke the SSVEP, gold cup electrodes placed on the channel Oz, A1 and Fpz, the Cyton acquisition device and the embedded bard. The electrodes were connected to the Cyton board to transform the EEG signals to digital data at a sample frequency (Fs) of 250 Hz. Also, Cyton board sends the EEG data by Bluetooth to an embedded board. The board is a Raspberry Pi that analyzes the EEG data with EP-SSVEP to find the stimulus that subject stares. Finally, the board sends by WiFi the message of the selected stimulus to an Internet of Things network (IoT) for any control or communication purposes.

Fig. 1.
figure 1

BCI embedded system scheme.

Full size image

For the experiments, a dataset was designed considering the regular organization of SSVEP datasets reported in literature [23,24,25]. Our dataset consists of eight healthy subjects that were selected from a group of male and female persons with an age from 21 to 25 years. All of them have normal or corrected-to-normal vision. None of them were taking medication. The total of EEG dataset signals is 384 (48 signals per subject).

3 Graphical Interface for Visual Stimuli Presentation

The common technology for the stimuli presentation to evoke the SSVEP is a digital monitor because the stimulation methodology can be designed with software, while other technologies would require hardware design.

To design the stimuli, there are two aspects to consider: figure and stimulation frequency.

The common figures are graphics (box, arrow, star, circle) and checkerboard inversion patterns [26]. However, according to the literature, a checkerboard inversion generates better amplitude response in SSVEP than figures because inversion causes the optical illusion that the checkerboard moves [23].

The stimulation frequency is calculated with the monitor refresh rate [3] and commercial monitors have 60 Hz of refresh rate, so the stimulus frequency fe is given as:

$$ f_{e} = {\raise0.7ex\hbox{${60}$} \!\mathord{\left/ {\vphantom {{60} Z}}\right.\kern-0pt} \!\lower0.7ex\hbox{$Z$}} $$
(1)

where Z is an entire number. Another aspect to be considered in frequency stimulation is that fe greater than 10 Hz can generate secondary effects in photosensitive subjects [26]. Hence, fe selection includes frequencies less than 10 Hz in Eq. (1).

Then, the interface for visual stimuli presentation is a graphical user interface with four black/white checkerboard patterns distributed as Fig. 2 shows. The fe of checkerboard are 6, 6.6, 7.5 and 8.5 Hz and they are distributed as Table 1 shows. The red points in the checkboard are used by the subject to stare the center of the selected checkerboard.

Fig. 2.
figure 2

Graphical interface for visual stimuli presentation. (Color figure online)

Full size image
Table 1. Frequencies distribution for checkerboards
Full size table

4 EP-SSVEP Method

This section presents the proposed method EP-SSVEP whose input is an EEG data recorded during 12 s from Oz EEG channel at 250 Hz. This signal is given by s(n), n = 0,…,2999, where n is the time index.

EP-SSVEP was developed with a machine learning approach because the SSVEP signals have noise and the amplitude frequencies may change in each session and subject. The modules of EP-SSVEP are preprocessing, feature extraction with DFT and CCA and classification with a FFNN as Fig. 3 shows. The output is the SSVEP frequency related with the fe that a subject stares.

Fig. 3.
figure 3

EP-SSVEP modules.

Full size image

4.1 Preprocessing Module

The preprocessing module removes the unnecessary frequencies from the EEG signal using a fifth order bandpass Butterworth filter with cut frequencies of 5.5 Hz and 9 Hz. The filter is expressed as H(ω): s → sf2, where H(ω) is the filter transfer function and sf2 is the output of the filter.

4.2 Feature Extraction

This module generates a feature vector to develop an efficient SSVEP detection by reducing the dimension of the frequency components of EEG signal. The first step of this module is the Discrete Fourier Transform given by:

$$ \begin{array}{*{20}c} {S\left( k \right) = \sum\limits_{n = 0}^{N - 1} {s_{f2} \left( n \right)e^{ - i2\pi kn/N} } } & {k = 0, \ldots ,N - 1} \\ \end{array} $$
(2)

where S(k) contains the frequency components of sf2(n), N = 1500. DFT is computed with the radix 2 fast Fourier transform algorithm. There are other methods to find the frequency components like SPD and DWT, but the DFT was selected because it finds a correct definition of SSVEP frequencies, the input is defined in one dimension and DFT reports the least processing time in feature extraction. The next step is a normalization defined by:

$$ x\left( k \right) = {{\left( {S\left( k \right) - \mu_{s} } \right)} \mathord{\left/ {\vphantom {{\left( {S\left( k \right) - \mu_{s} } \right)} {\sigma_{s} }}} \right. \kern-0pt} {\sigma_{s} }} $$
(3)

where µs is the mean of S(k) and σs the standard deviation. This normalization reduces the uncertainty generated by the amplitude levels of SSVEP signals. The final step is the Canonical Correlation Analysis, which was used because is one of the most popular methods to reduce the data dimension in SSVEP. CCA finds four Pearson correlation coefficients that represent similarity between x(k) and four SSVEP signals λj(k), j = 1,…4 at frequencies of 6, 6.5, 7.5 and 8.5 Hz. Signals of λj(k) are showed in Fig. 4, and they are the average of signals per subject obtained from training signals of dataset. Table 2 shows the distribution of SSVEP frequencies in each λj(k) signal. The CCA is defined as follows:

Fig. 4.
figure 4

Frequency signals λj(k).

Full size image
Table 2. SSVEP frequency distribution in λj(k).
Full size table
$$ \rho_{j} = {\raise0.7ex\hbox{${\sigma_{x\lambda j} }$} \!\mathord{\left/ {\vphantom {{\sigma_{x\lambda j} } {\sigma_{x} \sigma_{\lambda j} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sigma_{x} \sigma_{\lambda j} }$}} $$
(4)

where σxλj is the covariance between x(k) and each λj(k) signal, σx is the standard deviation of x(k) and σλj is the standard deviation of λj(k). Thus, the result is a correlation vector ρj = {ρ1, ρ2, ρ3, ρ4}, where each coefficient is the Pearson correlation between x(k) and each λj(k) signal. According to the definition of (4), if SSVEP is not presented in x(k), then, the values of ρj are close to zero, but if SSVEP is presented, the values of ρj have results in the interval from 0 to 1, and they vary according to the SSVEP frequency in x(k) and its quality. The frequency of SSVEP generates a value close to one in the component of ρj related to the λj(k) signal with the most similar frequency to the SSVEP frequency of x(k). However, the quality of the signal causes uncertainty in the results of ρj and that quality depends of the conditions of the subject during the BCI sessions. Therefore, the best option in classification module is a supervised algorithm.

4.3 Classification

According to experiments and the literature, classifiers with the best results in SSVEP analysis are artificial neural networks. Then, the classification module in this method is a Feedforward Neural Network (FFNN) trained with scaled conjugate gradient backpropagation algorithm. Figure 5 shows the architecture of the network, which includes three layers: input, hidden and output.

Fig. 5.
figure 5

FFNN architecture.

Full size image

The input is the vector ρj. The second layer has five neurons defined as follows:

$$ \begin{array}{*{20}c} {o_{m} = f_{1} \left( {\sum {w_{jm} \rho_{j} + b_{1m} } } \right)} & {m = 1, \ldots ,5} \\ \end{array} $$
(5)

where m is the neuron index in the hidden layer, wjm are the weights of the neurons, b1m is the neuron bias and f1(u) is the softsign activation function f1 (u) = u/(1 + |u|), where u is the input of the function. The number of neurons and f1(u) were defined by experimentation. The output layer has four neurons fully connected to om. This layer finds the stimulus frequency selected by the subject and the neurons are given by:

$$ y_{r} = f_{2} \left( {\sum {w_{mr} O_{m} + b_{2r} } } \right)\;r = 1, \ldots ,4 $$
(6)

where r is the neuron index in the output layer, wmr are the weights of the neurons, b2r is the neuron bias and f2(.) is a softmax activation function that generates the best results in the experiments. The output layer has four neurons because each neuron represents each frequency fe.

4.4 Frequency Stimulation

After propagation of ρj in the network, EP-SSVEP determines the selected fe with:

$$ e = \begin{array}{*{20}c} {\mathop {\arg \hbox{max} }\limits_{r} } & {y_{r} } \\ \end{array} $$
(7)

where e is the index value of r, which is the stimulus that the subject stares during the SSVEP session.

5 Results

This section presents the performance of EP-SSVEP and a comparison of this method with others reported in the state of the art.

EP-SSVEP and its training were implemented in a Raspberry Pi 3B model using Python 3.4 and the machine learning library TensorFlow™. The processing velocity average of EP-SSVEP is one frame per second. The Raspberry stores in the EEPROM the coefficients of H(ω), the trained weights per subject of the FFNN, the signals λj(k) and σλj. The total data stored in EEPROM is 171 Mb, which corresponds to 1.06% of total capability. During the processing of SSVEP, RAM memory stores s(n), sf2(n), x(k), sf2(n) signals, σx, ρj and the propagation of the network. The total data stored in RAM is 41 Mb, which corresponds to 4.7% of total capability.

Dataset described on Sect. 2 was divided in a training set that has 28 signals from 8 subjects, and a test set that has 20 signals from 8 subjects. The signals for training and test was randomly separated by cross-validation with five k-fold iterations. Figure 6 shows the confusion matrix of both sets and the reported performance in these matrices are 98.23% for training set and 96.09% for test set. Additionally, stimulus of 7.5 and 8.5 Hz (up and down) reports the best performance than stimulus 6 and 6.6 Hz (right and left) due vertical checkerboards were placed closer of central area of subject vision than horizontal stimulus.

Fig. 6.
figure 6

Matrix confusion generated with 5 k-fold cross validation for (a) training set. (b) Test set.

Full size image

5.1 Comparison with Other SSVEP BCI Systems

To evaluate the performance of EP-SSVEP BCI system, this section presents a comparison with other popular SSVEP BCI focused to embedded applications reported in literature. We do not found a database or methodology for comparisons of BCI embedded systems, thus we design a criterion considering the mainly aspects of BCI operation, which are:

Number of Subjects in the dataset (NS). EEG signals are noisy and they have a response that can be different in each subject. Then, in order to find a better statistical description of the method, it is important to increase as much as possible the number of subjects used for dataset and experiments.

Number of EEG channels (NE). The number of EEG channels is related to the computational cost of the processing methods, i.e., as the number of electrodes reduces, the complexity of the proposed method decreases. Additionally, a large number of electrodes can cause discomfort in the subjects, which affects the performance of the SSVEP sessions. Then, is important to reduce as much as possible the number of EEG channels without compromising the signal quality.

Accuracy (Acc). There are many metrics to compare SSVEP processing methods like Information Transfer Rate (ITR) [24], Accuracy (Acc) [2], etc. Among them, Acc is the most used and it refers to the number of SSVEP samples correctly detected divided by the total number of samples.

Table 3 shows the comparison of EP-SSVEP with two state of the art methods for specific purposes [22, 27] and other popular methods implemented in embedded systems [4, 28]. According to the results showed in Table 3, EP-SSVEP has better Acc than the other methods in spite of EP-SSVEP uses a single EEG channel and that method was tested with eight subjects. This Acc is because of the next reasons:

Table 3. Comparisons between EP-SSVEP and other popular methods.
Full size table

  • Subjects reports better concentration during the SSVEP sessions in our BCI system than other systems because our system is comfortable for subjects due to two reasons: the use of just one EEG channel and the card processor allows an ubiquitous embedded system.

  • The gold cup electrodes acquire the EEG signal with less noise than other commercial BCI.

  • EP-SSVEP needs data from one EEG channel and uses a classifier that is trained considering the natural uncertainty in SSVEP signals.

6 Conclusions

This paper proposes EP-SSVEP, a novel method designed for ubiquitous BCI embedded systems that process SSVEP signals from a single EEG channel. The proposed method is implemented in an embedded board that sends the information of stimulus that the subject stares to an IoT network. A dataset was designed for experimentations and it consists of the EEG signals obtained from eight subjects. EP-SSVEP is composed of three modules: preprocessing to eliminate noise of EEG signal, feature extraction that uses DFT and CCA and a classification with a FFNN. EP-SSVEP reports the best performance in vertical stimulus than horizontal because of the colocation of the checkerboards respect to the user vision. In the comparisons, EP-SSVEP is a method that generates a BCI embedded system that uses the least number of EEG channels and reports the best Acc. Then, based on these results, we conclude that EP-SSVEP is one of the best methods proposed in the literature. Additionally, EP-SSVEP allows the design of ubiquitous BCI embedded systems because this method can be processed in one second in embedded processors, includes the best accuracy in comparisons of Table 3 and uses as input one EEG data channel.