Keywords

1 Introduction

The MW is closely related to the operator’s professional knowledge, personality, task type and physiological variables [1]. Excessive MW can lead to rapid fatigue, distraction, and reduced efficiency, which result in errors in analysis and decision-making tasks. But too low MW can lead to waste of human resources [2].

The existing main methods of MW measurement include subjective measurement, performance measurement and psychophysiological measurement [3]. EEG refers to the recording of the electrical activity originated by brain [4]. Most EEG research methods divide EEG signals into four rhythms according to frequency band, including Alpha, Beta, Theta, Delta [5], and some studies have shown that the energy information in EEG is an important feature that can be used for classification [6].

EEG signals are non-stationary [7], the signals will change greatly due to the physiological changes of operators, environmental changes, and instrument changes [8]. When the distribution of data changes, most statistical models need to be rebuilt from scratch using newly collected training data. it is expensive or impossible to recollect the needed training data and rebuild the models [9].

Therefore, this paper proposes a MW classification method based on EEG Cross-Session Subspace Alignment (CSSA), which can build a personalized model for the mental load of new sessions, make full use of the existing session data, and improve the classification accuracy.

2 Experiments and Data Preprocess

2.1 Experiments

A 32-channel Neuroscan Neuamps (Synamps2, Scan4.3, EI Paso, USA) system is shown in Fig. 1, which can record all EEG signals. M1 and M2 are the reference electrodes of the system, which are located at the mastoid position behind the left and right ears respectively. EEG raw data is digitized at 1024 samples per second. Band-pass filter settings were 0.1–200 Hz [10].

Fig. 1.
figure 1

Channel location

Full size image

EEG signals were collected from 13 subjects in Beihang University (aged between 22 and 25 years, with 2 females and 11 males). The Multi Attribute Task Battery (MATB-II, National Aeronautics and Space Administration (NASA), USA) was used as the platform of experiment. Its task interface is shown in Fig. 2. The four regions correspond to some visual and operational missions respectively. During the experiment, two levels of MW were set, and they are low mental workload (LMW) and high mental workload (HMW).

The difficulty of the experimental task is defined by the occurrence frequency of the four sub-tasks. In the LMW task, 1 task occurred in 1 min, and in the HMW task, 24 tasks occurred in 1 min.

Fig. 2.
figure 2

MATB-II tasks

Full size image

2.2 Data Preprocess

The average values of M1 and M2 were used as the reference of data processing. The 30-channel EEG signals were filtered with 1–30 Hz finite impulse response (FIR) band-pass filter, and the filter order was 3300.

To ensure the reliability of data analysis, the EEG signals of 10 participants (sub1~sub10, aged between 22 and 25 years, with 2 females and 8 males) were selected to analyze according to the selection criteria of high performance and less artifacts [11].

The power spectral density (PSD) at different frequencies is often used to extract the features of EEG signals in the frequency domain [12]. Fourier transform in Formula (2.1) was performed on each data segment to obtain Fs(n).

$$ F_{{\text{s}}} (n) = \sum\limits_{n = 0}^{N - 1} {Se}^{{\frac{ - 2\pi j}{N}n}} $$
(2.1)

where S is the ICs of a group of EEG signals and is the Fourier transform of the signal. N is the number of signal samples. The value of n is 0 to N − 1.

The corresponding PSD of EEG ICs was calculated with Formula (2.2).

$$ p_{s} (n) = \frac{{F_{s} (n)F_{s}^{*} (n)}}{N} $$
(2.2)

where ps(n) is PSD of EEG ICs; Fs*(n) is the conjugate of Fs(n). N is the length of signal.

3 Classification for Cross Session Visual Tasks

3.1 Classification Method Based on CSSA

The method is divided into two parts. The first part is data preprocessing, which includes obtaining the independent components of EEG through ICA and extracting the energy features of ICs. The second part is domain adaptation, which includes subspace and dimension reduction features obtained with PCA algorithm, transfer features obtained with SA and MW classifier construction.

Before obtaining the EEG signals of the new session, we have obtained the EEG signals identified from session 1 to session N. We preprocess the known data separately and take the energy characteristics of EEG signals in each SA as a source domain. We do the same preprocessing to the unlabeled new session EEG signals and take the energy characteristics of the new session EEG signals as the target domain. The we aligned the subspace between the source domain and the target domain by SA method, the transfer domains were the training sample to construct the classification model.

3.2 SA Method

Let X ∈ x be the EEG recorded of a sample (X, y), where x = RC × D, where C is the number of channels, D is the number of samples of time series, in this paper, C = 30, D = 300. Y ∈ Y represent the corresponding MW label. Let P(X) be the marginal probability distribution of X, and P(Y|X) be the conditional probability distribution. We represent the source domain as XSi, where XSi = xSi, 1, ySi, 1,…, xSi, j, ySi, j, …, (xSi, D, ySi, D)}, i = 1, 2, …N, j = 1, 2, …, D, Similarly, the target domain is represented as XT, where XT = xT, 1, yT, 1,…, xT, j, yT, j, …, (xT, D, yT, D)}, j = 1, 2, …, D, Where (XSi, XT) ∈ X is the normalized relative power spectral density. Although both source domain data and target domain data are in d-dimensional space, their marginal distributions are different, while their conditional distributions are almost the same. We assume P(XS) ≠ P(XT), but P(YS|XS) = P(YT|XT), which means there are differences between different domains. The purpose of domain adaptation is to make P(ϕ(XS)) = P(XT) with a linear mapping ϕ.

According to PCA theory, we find the eigenvectors corresponding to the l largest eigenvalues in each domain. These eigenvectors are defined as source domain subspace and target domain subspace, which are represented as ZS and ZT respectively. Assuming that there is a linear relationship between ZS and ZT, we can find a linear mapping to align the source subspace with the target subspace. We use the transformation matrix S from ZS to ZT to align the basis vectors. The matrix S transforms the source subspace coordinate system into the target subspace coordinate system by aligning the source and target basis vectors. The transformation matrix S is achieved by minimizing the following Bregman matrix divergence:

$$ F(S) = \min ||Z_{S}S - Z_{T}||_{F}^{2} $$
(3.1)

where \(| \cdot ||_{F}^{2}\) is the F-norm. Because of the orthogonality of the F-norm, the above formula can be rewritten as:

$$ \min ||Z_{S}^{{ \top }} Z_{S}S - Z_{S}^{{ \top }} Z_{T}||_{F}^{2} = \min ||S - Z_{S}^{{ \top }} Z_{T}||_{F}^{2} $$
(3.2)

It can be seen from the above formula that when S* = ZSTZT, the minimum value of the above formula is obtained. For the coordinate system of the source domain, it can be written as:

$$ X_{a} = Z_{S}Z_{S}^{{ \top }} Z_{T} $$
(3.3)

By calculating Formula (3.4), we can get the representation of the source domain and the target domain in the new coordinate system after subspace alignment. Currently, the target domain and the source domain have the most similar marginal distribution.

$$ \begin{aligned} & G = X_{S}X_{a} \\ & H = X_{T}Z_{T} \\ \end{aligned} $$
(3.4)

4 Results of Classification Method Comparison

4.1 Comparison of Mental Workload Classification Using CSSA and not Using CSSA

MW Classification accuracy using CSSA and not using CSSA will be compared in this section. The former presented in this paper is named as Method 1, In method 2, the energy features of EEG ICs are used, PCA method is directly used to reduce the dimension of data instead of domain adaptation method. SVM classifier is used to obtain the MW classification results.

Through the experiment, EEG signals of 10 subjects lasting for one week were obtained and preprocessed. In this section, Session 0 represent the first day, and the data of this session is only used as training data. From the second day to the seventh day. Sessions 1 to 6 represent the second to seventh days. From the second day, the preprocessed energy features of the previous day are used as the training set, and the features of the day are used as the test set to construct the classification model.

The comparison of classification accuracy of the two methods is shown in Fig. 3. For the same time and data, the accuracy of Method 1 ranged from 73.3% to 88.1%, and the average classification accuracy was 80.2%. The accuracy of Method 2 ranged from 50.2% to 63.3%, and the average classification accuracy was 52.9%. Compared with method 2, the accuracy of method 1 is improved by 23% to 31.5%, with an average of 27.3%.

Fig. 3.
figure 3

Classification accuracy comparison of Method 1 and Method 2

Full size image

4.2 Comparison of Mental Workload Classification Using CSSA and TCA

MW Classification accuracy using CSSA and using TCA will be compared in this section. The same experiment as in Sect. 4.1 was completed.

The EEG signals were preprocessed in the same way, in the domain adaptation part, the accuracy of SVM classifier constructed by the proposed method and TCA method is compared. The comparison of classification accuracy of the two methods is shown in Fig. 4.

Fig. 4.
figure 4

Classification accuracy comparison of CSSA and TCA

Full size image
  1. (1)

    The accuracy of CSSA is between 73.3% and 88.2%, and the average accuracy is 80.2%.

  2. (2)

    The accuracy of TCA is between 66% and 87%, and the average accuracy is 74%.

The average classification accuracy of CSSA is higher than that of TCA, this is because the TCA method only uses the shared features in the two domains. The difference is that CSSA uses the relevant features of the two domains. On the other hand, the stability of CSSA method is better than TCA method. This is different from TCA method in dealing with the original data, CSSA method processes its subspace, which makes the algorithm more robust. Therefore, we choose CSSA method as the method of cross time MW classification.

5 Conclusion

In this study, we propose a MW classification method based on EEG cross-session subspace alignment, it includes filtering EEG signals, obtaining ICs by ICA method, extracting energy features, obtaining subspace according to PCA theory, aligning the source subspace base vector with the target subspace coordinate system base vector by learning a linear mapping.

The conclusions are as follows:

Compared with Method 2, the classification accuracy of the proposed CSSA method is improved from 52.9% to 80.2%. Compared with other domain adaptive methods, the proposed CSSA method has 6% improvement in classification accuracy compared with TCA method and can introduce multi-source domain to further improve the classification accuracy.