Keywords

1 Introduction

Visual attention is an important set of cognitive operations, which can filter out irrelevant information and select relevant information. In the learning process, whether a person being attentive or inattentive throughout instruction usually affect their learning efficacy [1]. Attentive behaviour can allow a person to stay on the task and acquired relevant information, while inattentive behaviour can cause a person to be unable to focus on the task. Inattentive occur due to external factors such as emotional stress and distraction. In academics, the teacher usually observes student’s expressions during learning to determine their mental state. However, it is a burden for a novice teacher to teach and monitor all the student expressions at the same time. To effectively determine a person’s mental state during the learning process, the use of EEG has become quite popular among the researchers. Tóth et al. [2] using the BrainAmp DC 64-channel EEG system to record the brain activity and study focusing attention and divided attention of the speech stream and the processing of speech under different time-scale and depth. Shestyuk et al. [3] used 32 active electrodes biopotential system and Biosemi ActiveTqwo DC-coupled amplifier to record the EEG signal of an individual. The signals were analysed using the power spectrum of the frequency bands to predict the population success rate of various TV shows and determine the cognitive processes (attention, memory, and motivation) that help with audio engagement with television shows. Aliakbaryhosseinabadi et al. [4] using 18 active electrode g.GAMMAcap system and g.USB amplifier to record the signals and find the time-frequency features to identify the user’s attention variation during the motor task execution. Although the use of research-grade EEG systems with many channels can obtain more features from the recorded brain signal, the device is expensive and the set-up is quite tedious, less versatility outside the laboratory environment, and uncomfortable to wear by the user. Recently, there has been a use of consumer-grade EEG device for analysis of the mental state of a person. Tan [5] using the one channel NeuroSky Mindset to record the brain signal and detect the mental state during the word trials. Van Hal et al. [6] using the one channel NeuroSky Mindset to record the signal and then filtered it into alpha and beta frequency bands to detect the onset of sleep.

In this study, we use the Emotiv Insight to record the EEG signals during the participant performing the Visual Attention Colour Pattern Recognition (VACPR) test. Emotiv Insight is a device designed for daily usage with an advanced electrode that uses 5 channels to record the brain signal [7]. The recorded raw EEG signal was then passed through pre-processing such as Notch filter and Independent Component Analysis (ICA) to filter out artifact noise. Next, the Power Spectral Density (PSD) was used to calculate the power value of the pre-processed signal to verify whether the signal is consistent with the mental state stimulated during the VACPR test [8]. After that, a Convolution Neural Network (CNN) was used to perform the classification on the signals. CNN is a deep learning algorithm that takes input signals, learns the features through the convolutional layer, and performs the classification of attention level in a fully neural layer [9]. The aim of this paper is divided into two parts. First is to verify whether the collected EEG signal using Emotiv Insight is consistent with the mental state stimulated during the VACPR test (task A and task B). Second is to determine whether the accuracy of the EEG classification on the tasks is higher than the chance (50% for binomial problem) [10].

Section 2 of this paper describes the material and methods used for performing the experiment, collecting the brain signals, and analysing the signals. Section 3 shows the result obtained from the experiment and its discussion. Section 4 is a conclusion that highlights the finding in this research work.

2 Materials and Methods

2.1 Participants

The experiment was conducted on fifteen participants (10 males and 5 females; age from 21 to 26). All participants voluntarily participated in the experiment and they are all healthy. A consent form was given to them before participating in the experiment. They were told that if they begin to feel unwell, they could stop the experiment. The experiment was performed for 4 weeks; in which they participated 1 day per week.

2.2 Consumer-Grade Electroencephalogram Device

The electroencephalogram device that used in this research is the Emotiv Insight headset shown in Fig. 1(a), which is a low-cost consumer-grade device designed to record daily brain activity. It consists of 5 channel electrodes with a 128 Hz sampling frequency. To determine the quality of EEG signals is under good condition, Emotiv Xavier Control Panel software was used as shown in Fig. 1(b). The software was used to check if the electrodes have properly touched the scalp of the head, as follows:

  1. (a)

    Black colour indicates no signal received. The electrode may not yet touch the scalp.

  2. (b)

    Red colour indicates bad signal quality. The electrode needs to adjust properly

  3. (c)

    Orange colour indicates poor signal quality. The electrode needs to adjust properly

  4. (d)

    Green colour indicates good quality. The electrode seems to be placed properly.

Fig. 1.
figure 1

(a) Emotiv Insight device and (b) EEG channel position

Full size image

Emotiv Insight is using a polymer-based semi-dry electrode, which is comfortable and easier to wear by the participant. The set-up time is only 2 min. Besides, the device works wirelessly, which means that you can easily connect to the computer or smartphone to record the EEG signals. Table 1 shows the parameters of the headset.

Table 1. Parameters of Emotiv Insight
Full size table

2.3 Experimental Design

In this research, the visual attention colour pattern recognition (VACPR) test was designed and used as shown in Fig. 2. VACPR test is a type of perceptual test that requires visual attention to scan the environment and search for the targeted object. The design and concept of VACPR test are refer from PsyToolkit, a website that running cognitive-psychological experiments that maintained by a physiologist [11]. The use of the VACPR test can stimulate participants to be attentive or inattentive in a relatively short time. This can be done by dividing the test into 2 tasks (task A and task B).

Fig. 2.
figure 2

Stimulus presented on each trial in the Visual Attention Colour Pattern Recognition (VACPR) test

Full size image

For task A shown in Fig. 2(a), the purpose is to stimulate participants to be attentive. Participants were initially prompt for the targeted stimulus (TS) with random colour (red or blue) and pattern (vertical and horizontal) in each trial. Later, the participant needs to search for TS in the stimulus display (SD) filled with non-targets. The colour of all non-targeted stimulus is the same as the targeted stimulus, but with a different pattern. A similar concept also applies to task B shown in Fig. 2(b). The purpose of task B is to stimulate participants to be inattentive by including distractors and inducing emotional stress. Participants were initially prompt for the TS with random colour and pattern in each trial. After that, a distractor (same colour with different pattern, different colour with the same pattern, and different colour with different pattern) appeared in SD and the participant needs to find targeted stimulus among distractors. Figure 2(c) shows the stimulus sequence for task A and task B, in which duration of 500 ms will be given to the TS and the SD for each trial.

To collect the EEG signal when the participant performing the VACPR test, an experiment protocol was constructed as shown in Fig. 3. The duration of the experiment is about 180 s. Before the experiment, the participant was informed about the nature of the experiment. Then, the participant was given an instruction sheet that explains the experiment process. Later, the participant entered a closed room with a chair, table, computer, and EEG device prepared in advance. First, the participant wears the Emotiv Insight and rest for 30 s before starting to record. After that, the participant begins recording 60-s of task A. After finishing the first task, the participant was allowed to rest for 30 s. Afterward, the participant begins recording 60-s of task B. All recorded raw EEG data during the test was saved in the computer for further analysis.

Fig. 3.
figure 3

Experiment protocol

Full size image

2.4 EEG Signal Pre-processing

Since the raw EEG signal contains unwanted noise such as eye blinking, muscle movement, and line noise, there is a need to perform the pre-processing on the signals [12]. Figure 4 shows the pre-processing technique used in this research.

Fig. 4.
figure 4

Block diagram of EEG signal Pre-processing Technique

Full size image

To remove the electrical line noise from the raw EEG signal, a digital notch filter is implemented. This filter filtering out the noise found at 50 Hz and 60 Hz.

$$ G\left( x \right) = \frac{{1 - 2\cos \left( {50} \right)x^{ - 1} + x^{ - 2} }}{{1 - 2r\cos \left( {50} \right)x^{ - 1} + r^{2} x^{ - 2} }} $$
(1)

where x is the z-transform of raw EEG signal. Next, to eliminate the eye blinking and muscle movement, an Independent Component Analysis (ICA) is employed. The use of ICA is to separate mixture signals into their respective sources, thus can easily identify the noise in the signal. Equations 2 and 3 show the formulation of ICA.

$$ S = Vx $$
(2)
$$ V = \left[ {\begin{array}{*{20}c} {\cos \emptyset } & {\sin \emptyset } \\ { - \sin \emptyset } & {\cos \emptyset } \\ \end{array} } \right]y^{ - 1} \left[ {\begin{array}{*{20}c} {\cos \theta } & {\sin \theta } \\ { - \sin \theta } & {\cos \theta } \\ \end{array} } \right] $$
(3)

where S is the independent component, V is the matrix of independent component value, and x is the EEG input signal.

2.5 EEG Power Spectral Density

In this study, out of 5 frequency bands (delta, theta, alpha, beta, and gamma), only beta (13–30 Hz) are interested. Beta wave is a high frequency that is commonly observed in an awake state. It is responsible for conscious thinking and the active processing of information [13]. The reason for using the beta band in this research is because its power value can provide information about the participants’ visual attention when performing the task. To measure the power value of the beta band, the Power Spectral Density (PSD) was used. The PSD contained in the beta band is extracted from the Fast Fourier transform (FFT) as shown in Eq. 4

$$ P\left( f \right) = \mathop {\lim }\limits_{N \to \infty } \frac{1}{N}\left| {T\left( f \right)} \right|^{2} $$
(4)

where \(P\left( f \right)\) is the spectral power and \(T\left( f \right)\) is the Fourier transform that converts the signal from a time domain to a frequency domain.

2.6 Convolution Neural Network

In this research, the deep learning algorithm used is a Convolutional Neural Network (CNN) shown in Fig. 5, which uses pre-processed EEG signal as the input, learns important features from various aspects of the image, and distinguishes them from each other in the neural network [14].

Fig. 5.
figure 5

The architecture of CNN

Full size image

From the architecture, the EEG signal is initially converted into a 2-dimensional grayscale image and used as the input of CNN. Since the EEG signal exhibit significantly complex behaviour with dynamic and non-linear characteristics, 2 blocks of convolutional layer and pooling layer are used. In the convolutional layer, the image will go through convolution operation and using the kernel to extract the feature maps. Low-level features will be extracted by the first convolutional layer, but as layers are added, the architecture will slowly learn and extracted the high-level features. Besides, rectified linear unit (ReLu) is also used in the convolutional layer as an activation function that helps to speed up the training. For the pooling layer, it is used to reduce the size of the features. The purpose is to decrease the computational power required to process the data and obtain the significant features among the feature maps. Next, all the extracted features will be flattening and go through a fully connected layer to classify the learned feature into 2 classes which are attentive and inattentive. Furthermore, hyperparameter tuning has been done for CNN. First is the learning rate where the value is set to 1 × 10–4 to adjust the weight in network. Second is the epochs where the value is set to 20 to achieve a small gap between test error and training error. Third, the batch size is set to a value of 10. Fourth, a dropout is used to avoid the overfitting to the data in network.

3 Result and Discussion

3.1 Power Spectral Density in Beta Band

To verify that whether it is sufficient to use Emotiv Insight to record brain activities when participant performing the Visual Attention Color Pattern Recognition (VACPR) test, the Power Spectral Density (PSD) values of the low beta band and the high beta band for frontal and parietal channels (AF3, AF4, and Pz) are averaged and compared, as shown in Fig. 6. According to the study [15], a low beta band (12–20 Hz) is associate with focus and concentration, while a high beta band (20–30 Hz) is associate with significant stress and distraction.

Fig. 6.
figure 6

Power spectral density in low and high beta bands

Full size image

Figure 6 shows the PSD value obtained in 2 different tasks. Task A stimulates the participant to be attentive, while task B stimulates the participant to be inattentive. Based on Fig. 6(a), we can see that when the participant performing task A, the PSD value of the low beta band is greater than the PSD value of the high beta band, indicating that the collected EEG signal shows the participant is focusing on the task. As for Fig. 6(b), when the participant performing task B, the PSD value of the high beta band is greater than the PSD value of the low beta band, indicating that the collected EEG signal shows the participant is distracted and might emotionally stress. From the result, we can verify that the collected EEG signal using Emotiv Insight is satisfied the collectability requirement as the PSD value of the collected EEG signal is consistent with the mental state stimulated during task A and task B.

3.2 Accuracy Using Convolutional Neural Network (CNN) Classifier

After verifying that the use of Emotiv Insight in the VACPR test can provide significant information about the participant brain activity, the next step is to determine how accurate the classification of collected EEG signals in the VACPR test using the Convolutional Neural Network (CNN). Figure 7 shows the progress of training and testing accuracy.

Based on the graph, we can see that the EEG classification of VACPR tasks (task A and task B) can reach 76% accuracy, which is much better than chance (50% for binomial problem). This indicates that Emotiv Insight with CNN can provide good performance in attention level classification.

Fig. 7.
figure 7

Training and testing accuracy curve over 20 epochs

Full size image

4 Conclusion

This research has presented some findings that involve the use of Emotiv Insight to capture the brain activity of participants and the use of Convolutional Neural Network (CNN) to classify attention level based on the collected EEG data. The Power Spectral Density (PSD) result shows that Emotiv Insight is reliably in capturing brain activity since the PSD value of the collected EEG signal is consistent with the mental state stimulated during task A and task B. For the EEG classification of 2 tasks, accuracy up to 76% was achieved which is higher than chance. Therefore, it is feasible to use Emotiv Insight with CNN for attention level classification.