Classification of Imagined Digits via Brain-Computer Interface Based on Electroencephalogram

Abstract

Brain-Computer Interface (BCI) aims to improve the detection and decoding of brain signals acquired by electroencephalogram (EEG). In the recent years, artificial intelligence development has stepped onto a new stage, which boosts the research of BCI. This paper focuses on implementation of BCI by recognition of imagined digits. The subject was asked to imagine a digit 0 or 1 without other stimulation. The experiment was conducted using a 14 electrode Electroencephalogram. The subject was asked to imagine a digit for 30 s and the signals were recorded for analysis. Based on the results of the classification and ERP analysis, the O1 and O2 electrode positions (10–20 system) were chosen. Four methods were proposed for the feature extraction by using Event Related Potential (ERP) analysis, Power Spectral Density Analysis (PSD), Independent Component Analysis (ICA) and Common Spatial Pattern (CSP). Finally, several classification methods were used to recognize the imagined digits based on the extracted features. Experimental results showed that the results obtained by Artificial Neural Network (ANN) after CSP performed the best. The classification accuracy achieves 66.88%.

1 Introduction

A Brain-Computer Interface (BCI) system enables biological signals from the brain to be recognized by the computer. In some applications, the subject may be able to give commands to devices connected to the computer. A popular implementation of BCI is a wheelchair controlled by a signal acquired from EEG. However, the implementation of BCI using EEG is not limited to the field of biomedical engineering for medical purposes. BCI is also popular for gaming and further application in virtual reality or augmented reality (VR, AR) [1] and even for improving the traditional education system by understanding the emotions. In [2], the author proposed the method which points out that human-computer interaction takes part in emotion building. This is important to show the relationship between emotion and learning.

In this paper, we propose a novel method for binary classification of imagined digits with the intention to enable more classification methods for the purposes of medical and other applications of BCI. It can also be used for supporting behavior learning as the measurement is done for cortex of human brain. Beard in [3] found that there is a significant effect of students can change the emotions of students’ engagement in learning. By using the imagination recognition system, the emotion control system for learning in school can also be assisted.

For feature extraction from the EEG signal, Event Related Potential (ERP) and Common Spatial Pattern (CSP) are combined together. In addition to ERP, other EEG measurement methods such as Visual Evoked Potentials (VEP), Steady-State Visual Evoked Potentials (SSVEP), Event Related Desynchronization/Synchronization (ERD/ERS) can be used for the measurement. Due to the scenario of imagining the digit, unstimulated by the environment, we found that ERP analysis worked the best in this case. ERP enables the extraction of information from the EEG signals which are related to a certain motor event, cognitive, or specific sensory stimuli [4].

In this experiment, the ERP analysis was done for the whole group (of 10–20 system) electrodes used in the experiment. The ERP analysis was done by plotting the potential value on each epoch. One epoch is a 10 s span starting at 10 s and ending at 20 s within a 30 s recording. The ERP plot would then show the location of electrodes related to the events. Power Spectral Density (PSD) computation was then applied for removing bad electrodes and checking the amplitude of certain frequency ranges for the improvement of classification rate. PSD itself in this case is defined as the Discrete Time Fourier Transform (DTFT) of the covariance sequence [5].

After the ERP analysis and before the classification process, as a part of the classification method, Common Spatial Pattern (CSP) was applied. CSP is normally applied on multivariate signal processing such as the Electroencephalogram signal. It will find the spatial filters which maximize the variance of a class corresponding to the other classes. The author in [6] introduced the CSP method which is used to estimate the ERPs as they are naturally contaminated by biological and instrumental artifacts. CSP uses the multivariate spatio-temporal filtering to increase the signal-to-noise ratio. Besides enabling BCI to recognize a certain event (hand or leg movement for example), CSP can also be used for artifact cancelation in EEG implementation.

This study is also compared to the researches by Seto in [7]. The experiment done by Seto shows a similar recognition process as we proposed in this paper, by having the imagination of four directions. The signal processing and classification by Seto, however, used the band-pass filter, Fast Fourier Transform, Normalization, Principal Component Analysis (PCA), and three layers neural network. The identification rate using the 14 channels EEG is 46%.

2 Material and Experiment Setup

• A. Electroencephalogram (EEG) Equipment

Emotiv EPOC+ is a commercial EEG, well known in practical EEG research. This EEG is a multi-channel portable system with a sequential sampling method. The system itself consists of 128 Hz sampling rate and 14 electrodes with 2 reference electrodes. The electrode position follows the 10–20 electrode positioning system in which they are: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4. Figure 1 shows the position of the electrodes corresponding to the naming in the 10–20 universal electrode positioning system [8].

Fig. 1.

Emotiv EPOC+ Electrode positions (Color figure online)

Before wearing the EEG, each electrode should be given saline solution such that the impedance can be reduced. After wearing the EEG correctly, TestBench^TM, a software for the calibration of Emotiv EPOC+ is used. The electrode positioning in Fig. 1. should be all in green so that the EEG is ready for recording.

• B. Experiment Setup

The experiment is meant to record the brain signal where subject was imagining binary digits (0 and 1). For the training dataset, there were 100 trials of experiments conducted for imagining 0 and 1 each. Within a single experiment, the subject was given 30 s of the whole single experiment time to imagine the digit 0 or 1. Before the experiment began, the subject was given a printed image of digit 0 or 1 such that there is a standard for the imagination process. After 30 s of recording, there were 20 s of resting time before the next recording round (Fig. 2). The recording for a single binary digit imagination recognition was 15 trials a day excluding the training time and low concentration in imagining trials.

Fig. 2.

Experiment recording time setup

3 Data Preprocessing

The recordings of all trials in this experiment were put into folders. The data format of Emotiv EPOC+ recordings is European Data Format (EDF) file. In the beginning of data pre-processing, all of the EDF files were read all together and concatenated. Then, raw data was plotted to check which electrode should exist and could be selected for implementation. Obviously, the 14 essential electrodes as depicted from the Fig. 1, excluding the references node, are the electrodes which should be selected for the experiment. From the raw data plot as given in the Fig. 3, there were several electrodes which were not essential for the classification implementation.

Fig. 3.

Raw data plot

After that, a set of bad channels was pointed out such as Counter, Interpolated, Raw_Cq, Gyrox, Gyroy, Marker, Marker_hardware, Sync, Time_Stamp_s, Time_Stamp_ms, CQ_AF3, CQ_F7, CQ_F3, CQ_FC5, CQ_T7, CQ_P7, CQ_O1, CQ_O2, CQ_P8, CQ_T8, CQ_FC6, CQ_F4, CQ_F8, CQ_AF4, CQ_CMS, and STI 014.

The essential 14 channels are marked in the black in the Fig. 3. plot while some other unessential channels such as Counter, Interpolated, Raw_Cq, Gyrox, Gyroy, and others are marked as grey (will not be used in the future). From Fig. 3, it can be seen that the signals were messy with noise interference. Therefore, the raw data consisted of the 14 channel raw EEG data which was then put into an Infinite-Impulse Response (IIR) band-pass filter within the range of 8–30 Hz [9]. The band-pass filter range was between 8–30 Hz because it followed the condition of the experiment scenario, including alpha and beta waves as given in Table 1. The transfer function of the IIR filter is shown in Eq. (1):

Table 1. EEG frequency and state

$$ H\left( z \right) = \frac{{\mathop \sum \nolimits_{i = 0}^{P} b_{i} z^{ - i} }}{{1 + \mathop \sum \nolimits_{j = 1}^{Q} a_{j} z^{ - j} }} $$

(1)

Where P is the feedforward filter order, b_i are the feedforward filter coefficients, Q is the feedback filter order, a_i are the feedback filter coefficients, and z is the symbol for applying the z-transformation.

As shown in Table 1, delta wave ranging from 1–3 Hz will have high amplitude fluctuation whenever a person is in the condition of deep sleep [10]. Hence, for many other frequency ranges such as theta, alpha, beta, and gamma; each band will have high potential within the specific frequency range and states. In the experiment, the band pass filter will be done for the range of alpha and beta waves because these two bands happen when a person is in an awake condition. While delta and theta waves may have higher fluctuation when a person is sleeping. The band pass filter result of the raw data is given in Fig. 4. Noise on the 14 channels was suppressed, and the desired data could then be processed.

Fig. 4.

The raw data plot of 14 channels after band pass filter

After band-pass filter, the raw data can be used for ERP analysis. Then, the ERP per channel was plotted to check whether a channel contained suitable data for classification or not.

Figure 5 shows the ERP plot of O1 and O2 channels. The vertical axis shows quantity of epochs. The black waveform on the horizontal axis of the ERP plot shows the average electrical potential (of all epochs) for that electrode in the time domain. The small dots show synchronization in red, desynchronization in blue. Otherwise, the region will be mostly white, showing that there is less event related to the reaction from the subject. The more red and blue dots inside the graph, the higher the electrical potential fluctuation due to the event stimuli.

Fig. 5.

The ERP Plot of O1 and O2 (Color figure online)

From the 14 electrode locations, 2 of them contain the most data correlated to the event, they are O1 and O2. To check the potential of noise on the chosen locations, Power Spectral Density (PSD) analysis was applied. A plot of PSD is shown in Fig. 6. It can be seen that there is a channel with high noise which should be considered as a bad channel, i.e. FC5. Based on the PSD analysis and after removal of FC5 from the PSD calculation, it can be seen that O1 and O2 showed the highest power in the range of 10 to 13 Hz. Then the data of O1 and O2 was filtered by a band-pass filter within the range of 10–13 Hz, as shown in Fig. 7. The filtered results of O1, O2 were both chosen for the classification.

Fig. 6.

PSD plot before re-filter

Fig. 7.

PSD plot after frequency selection

After choosing the electrode position and frequency for classification, the raw data was put into epochs. One epoch consists of one trial of imagining a digit. The time was limited within the range of 10–20 s per epoch before being concatenated. The events per epoch will be set as labels for classification. After epoching and labeling, the data was then processed by applying Fast Independent Component Analysis (Fast ICA) for a better classification result.

4 Classification and Analysis

There were several methods used for classification in this experiment for comparison reasons. The main machine learning classification algorithms were Support Vector Machine (SVM), Artificial Neural Network – Multi Layer Perceptron (ANN-MLP), Logistic Regression, and Linear Discriminant Analysis (LDA). However, in order to increase the classification rate, Common Spatial Pattern (CSP) was applied in SVM, ANN-MLP, and LDA. A different method for Logistic Regression was applied by using the xDawn Covariance and Tangent Space algorithm. The whole classification method is given in Fig. 8.

Fig. 8.

Data processing methods

All of the classification algorithms were evaluated using KFold validation with the parameter of 10. The result was determined by the confusion matrix, while the classification rate is determined by the accuracy with the formula:

$$ Classification\; rate = \frac{tp + tn}{tp + fp + tn + fn} $$

(2)

Where tp, tn, fp, and fn are true positive, true negative, false positive, and false negative.

The classification rate (the accuracy to classify the given label) of LDA after CSP was 60% with the confusion matrix shown in Fig. 9. The elapsed time using the 10-Fold cross validation for the LDA method is 0.541 s.

Fig. 9.

LDA confusion matrix

For SVM classification, the configuration of SVM itself was RBF kernel with the C value 1. SVM is the general learning method based on statistical system and it is effective to process nonlinear and high dimensional pattern recognition [11]. The classification rate of SVM after CSP was 66.25% with the confusion matrix shown in Fig. 10. The elapsed time using the 10-Fold cross validation for the SVM method is 0.621 s.

Fig. 10.

SVM confusion matrix

ANN-MLP classification algorithm in this scenario consisted of quasi-Newton solver method with hidden layers with 14, 10, and 5 number of perceptron respectively. The classification rate of ANN-MLP after CSP was 66.88% with the confusion matrix given in Fig. 11. The elapsed time using the 10-Fold cross validation for the ANN-MLP method is 2.167 s.

Fig. 11.

ANN-MLP confusion matrix

The last classification algorithm used in this experiment was the Logistic Regression Algorithm. However, in order to optimize the result of Logistic Regression classification algorithm, instead of applying CSP before Logistic Regression, xDAWN Covariance was calculated and then the Riemann Tangent Space was projected. This method was applied on P300 BCI [12]. The classification rate of Logistic Regression with xDAWN Covariance and Riemann Tangent Space was 65.62% with the confusion matrix shown in Fig. 12. The elapsed time using the 10-Fold cross validation for Logistic Regression method is 1.949 s.

Fig. 12.

Logistic Regression confusion matrix

After applying the all classification methods, the result of this experiment was compared with the data processing method of a similar imagination recognition experiment by Seto et al. in imagining direction. The data processing method proposed by Seto et al. did not select the electrode position with ERP and apply PSD analysis for feature selection and CSP for feature space enhancement. But as depicted in Fig. 13, the data was first band-pass filtered within the range of 8-30 Hz frequency including Alpha and Beta states, then the data from all electrodes would pass a band pass filter, be normalized, and processed by using Principal Component Analysis. After the pre-processing part, the data was classified by using a three-layer neural network and validated with 5-fold validation [13].

Fig. 13.

Yuki Seto et al. data processing

By applying the same data processing method proposed by Seto et al. for our experiment scenario, the classification rate yields 52.5% with the confusion matrix shown in Fig. 14. The elapsed time using the 10-Fold cross validation is 3.406 s.

Fig. 14.

Seto et al. method confusion matrix

The summary of the results classification rate for all data processing methods is listed in Table 2. It can be seen that the best classification method in this case is CSP+ANN MLP with the rate of 66.88%.

Table 2. Performance comparison of the four methods proposed in this paper and the method used in [12]

5 Conclusion

This paper has shown a novel method of classifying the human cognitive state when imagining the binary digit (0 or 1). For the feature selection, ERP analysis was conducted together with the PSD plot. Then, O1 and O2 from 10–20 electrode positioning system were chosen to give the best classification rate after they were filtered by a band-pass filter within the 10–13 Hz frequency range. The experimental results showed that the ANN-MLP after CSP process outperformed the other data processing methods. In a future experiment, the improvement of classification rate will be conducted by applying modern machine learning classification methods and more feature extraction methods such as Hjorth Parameter and Fractal Dimension, and Embedding Sequences Analysis. Furthermore, better cognitive state classification study will be applied using an EEG system with more electrodes. Additionally, the experiment using deep learning classification method with automatic feature extraction is being conducted.