1 Introduction

A mental workload (MWL) is nothing but the quantity of mental resources required for the completion of a task [1]. According to a previous study, errors could be caused due to mental overload during decision-making [2], one of the vital reason behind mistakes/accidents. Keeping workloads low all of the time, on the other hand, might prevent bad decisions, but it might results in the wastage of mental resources and cause low job efficiency [3]. Hence, an optimum mental workload level is necessary to ensure high efficiency while avoiding overloading. A precise determination of workload level is required for this [4].

Various physiological parameters like cardiac activity, brain activity, eye activity, speech measures etc., can be used to analyse the MWL condition [1, 5]. For example, in [6] authors investigated the relationship between mental workload and physiological reaction using non-invasive physiological parameter monitoring. Shao et al. explored the MWL identification in a human and dual-arm robot interaction task by extracting 20 types of features from (heart rate variability) HRV, including frequency-domain, time-domain, and nonlinear features [7]. For the MWL classification, Abhishek et al. obtained the multi-scale entropy based features of HRV [8]. Similarly, an approach for classifying mental workload using electrocardiogram (ECG) signals is presented in [9]. Electroencephalography (EEG) is comparatively better method for evaluating workload level than other neuroimaging tools because it reflects to the brain activity [10].

The use of brain connectivity analysis approaches to obtain clinical knowledge about the dynamics and structure of abnormal functional brain networks (FBNs) has been a current trend in brain functionality analysis using EEG data [11].

The examination of global and local elements of brain connection is possible with the complex network representation [12]. The architecture of these connections is linked directly to cerebral function, and it can be influenced by the performed tasks or even by psychiatric and neurological illnesses. In fact, FBN has been used to evaluate abnormal connectivity topologies between brain areas in a variety of disorders. The interaction between brain regions involved in motor imagery (MI) and movement execution has also been studied using functional connectivity [13].

Although various studies have considered the FBN-based brain functionality analysis [14], very few studies deal with the dynamic analysis of brain functionality for mental workload activities. The literature presents a graph metrics-based quantification of the temporal variations in the FBN. For example, the study [15] extracted dynamic graph-based metrics from EEG data during an experimental procedure to evaluate FBN with respect to time. However, this oversimplified approach neglects the weight information of the edges in the network. The structural analysis of the FBN for the mental workload analysis, which enables us to visualize the active brain regions and temporal variation in the FBN connectivity, is missing in the literature.

The proposed study aims to eliminate the aforementioned limitations by developing an advanced network identifier called a weighted edge ordinal sequence (WEOS), which characterizes the dynamic FBN corresponding to a different level of mental workload. The proposed identifier utilizes the strength information of connections and the ordinal relation between them. In addition, the proposed study provides a dynamic structural analysis of the FBN, which enables us to visualize the temporal variation in the FBN and to identify the active brain regions corresponding to different levels of mental workload. To the best of the authors' knowledge, the proposed study is the first to provide weighted ordinal edges-based network identifiers and the structural analysis of EEG-based FBN for dynamic mental workload estimation.

1.1 Contributions and organization of the paper

The contribution of the proposed study is 4-fold as follows:

  1. i.

    Development of an approach for Dynamic characterization of Weighted FBN (WFBN) connectivity structure for different level of mental workload condition.

  2. ii.

    Development of an in-depth WFBN analysis technique, which includes dividing a WFBN into multiple subgraphs, evaluating each subgraph independently, and finally integrating the knowledge from each subgraph for network characterization.

  3. iii.

    Proposed an effective network characterization technique using weight information of edges and their ordinal connectivity.

  4. iv.

    Quantitative and structural analysis of the obtained results is performed to validate the achieved performance.

The paper is organized as follows; Sect. 1 briefly introduces the mental workload and reviews existing studies corresponding to the cognitive workload identification. A detailed description of the proposed technique and materials used in the study are presented in Sect. 2. Section 3 provides systematic representation of the obtained results. Section 4 concludes the proposed study with some future directions.

2 Materials and methodology

This section discusses mental workload estimation using the dynamic weighted FBN based approach. There are four subsections included in this section, namely data acquisition & pre-processing (Subsection 2.1), weighted FBN construction (Subsection 2.2), subgraphs construction and WEOS extraction (Subsection 2.3), a framework for feature extraction (Subsection 2.4), feature vector construction (Subsection 2.5) and classification (Subsection 2.6). The flow diagram of the proposed approach is depicted in Fig. 1.

Fig. 1
figure 1

The flow diagram of the proposed approach

Full size image

2.1 Data acquisition & pre-processing

This “STEW” dataset has been used in the study [16] for the experimentation. The dataset consists of 48 subjects EEG data corresponding to the two types of experiments, i.e., “simultaneous capacity (SIMKAP) based multitasking activity” and “no task”. The “SIMKAP-based task” [17] is a multitasking activity based on ratings. The EEG data have been recorded using the portable EEG headset having 14 electrodes (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4) with two reference electrodes (CMS, DRL). The sampling frequency of the recording has been kept at 128 Hz.

Before testing the proposed approach on the dataset, certain pre-processing steps need to be performed. The EEG filtration is performed to eliminate the using bandpass filters to remove the artifacts. The frequency range between 4 and 32 Hz is set as a permissible band for the filter [1].

The experimentation starts with the dataset distribution into the training and the testing sets, where training set data is used for the construction of WEOS based framework for feature extraction. As the study aimed at the dynamic analysis of brain functionality, the segmentation of the EEG dataset is performed using a sliding window approach. Here, two different types of segmentation are considered for testing the proposed approach (a) 30 s long segment without overlapping (30S_0), (b) 50 s long segment without overlapping (50S_0).

2.2 Weighted FBN (WFBN) construction

The WFBN is constructed for each subject corresponding to each segment of the EEG data. The WFBN is modeled as G = {V, E, W}, where V depicts the set of nodes/vertices, and E depicts the set of edges. For the EEG-based WFBN, vertices represent the location of electrodes used for data recording. The edge between two nodes can be constructed by calculating similarity measures between EEG signals corresponding to the electrodes considered as nodes. The W represents the strength/weight of the connection, i.e., the value of the similarity measure for the edge. In this study, coherence[18] is considered as a similarity measure to construct the edge between two nodes. In this way, the WFBN is constructed for each multichannel EEG record corresponding to different segments. The FBNs belonging to the same category are grouped together. The group of FBNs belonging to the NO_TASK category is represented as \({T}^{-}\) and the group of FBNs belonging to the SIMKAP_TASK category is represented as \({T}^{+}\).

2.3 Subgraph construction and WEOS extraction

The WEOS-based feature extraction technique is built on the idea of subgraph mining. This aims to identify and differentiate the subgraph pattern obtained from different categories of EEG data. Here, four subgraphs, namely depth-first search (DFS)[31], breadth-first search (BFS)[32], Dijkstra’s shortest-path tree (DSPT)[33], and minimum spanning tree (MST)[34], are constructed for each WFBN, and the WEOSs are extracted from each subgraph. The WEOS represents the sequence of weighted edges with an ordered relation. The pseudocode for the WEOS extraction process is given in Algorithm 1.

figure a

2.4 Framework for feature extraction

The WEOS-based framework is constructed using the training dataset only. Construction of the framework is a three-step process: frequent WEOS extraction, discriminant WEOS extraction, and finally, the WEOS-based framework generation. Each of the mentioned steps is described below.

2.4.1 Frequent WEOS extraction

This step is used to obtain the list of WEOS, which are common in the most number of subjects belonging to the same group. The most frequent WEOS for a particular category of subjects is extracted based on the value of the frequency ratio calculated for each WEOS of that category. The mathematical formulation for the calculation of frequency ratio \(({F}_{r})\) for WEOS (\(ws\)) belonging to category \(T\) where \(T\in \{{T}^{-}, {T}^{+}\}\) is given in (1).

$${F}_{r}\left(ws|T\right)= \frac{1}{N}\sum_{n=1}^{N}{\partial }_{n}$$
(1)

Here, \(N\) represents the number of subjects in category \(T\). The value for \({\partial }_{n}=1\), if \(ws\) is the WEOS of WFBN \({G}_{n}\in T\) else \({\partial }_{n}=0.\)

The WEOS \(ws\) is considered the most frequent if \({F}_{r}\left(ws|T\right) >\theta\). Here, \(\theta\) represents the threshold value, which is selected based on the trial-and-error approach.

2.4.2 Most discriminative WEOSs

The most frequent WEOSs from Sect. 2.4.1 is further evaluated to find the most discriminative sequences. WFBN belonging to a group \({T}^{-}\) has a different connection pattern than WFBN belonging to a group \({T}^{+}\). Discriminative WEOSs represent this topological difference between different groups of subjects. A parameter called ratio score (\(R_{s}\)) is used to calculate the discriminative power of WEOS (\(ws \in T\).), as shown in (2).

$$R_{s} \left( {ws_{\varphi } {|}T_{\vartheta in} } \right)\, = log\frac{{\mathop \sum \nolimits_{n1 = 1}^{{N_{\vartheta in} }} {\Phi }_{{n1,{ }\vartheta in}}^{\varphi } }}{{\mathop \sum \nolimits_{n2 = 1}^{{N_{\vartheta ot} }} {\upbeta }_{{n2,{ }\vartheta ot}}^{\varphi } + \eta }} \times \frac{{N_{\vartheta in} }}{{N_{\vartheta ot} }}$$
(2)

Here, \(\vartheta in\in\) {\({T}^{-},{T}^{+}\)} represents the category of interest, and \(\vartheta ot\in\) {\({T}^{-},{T}^{+}\)} represents the other categories. While calculating the \({R}_{s}\) for \(ws\in {T}^{-}\), the category of interest is \({T}^{-}\), and the other category will be \({T}^{+}\), similarly, while calculating the \({R}_{s}\) for \(ws\in {T}^{+}\), the category of interest is \({T}^{+}\), and the other category will be \({T}^{-}\). \({T}_{\vartheta in}\) represents the group of WFBNs belonging to the \(\vartheta in\) category. \({N}_{\vartheta in}\) and \({N}_{\vartheta ot}\) represents the number of WFBNs belonging to the \(\vartheta in\) and \(\vartheta ot\) category, respectively. If \({ws}_{\varphi }\) is present in the \({n1}^{th}\) WFBN of category \(\vartheta int\) then the value of \({\Phi }_{n1, \vartheta int}^{\varphi }\)=1 otherwise \({\Phi }_{\mathrm{n}1, \vartheta int}^{\varphi }\)= 0. Similarly, the value of \({\upbeta }_{n2, \vartheta ot}^{\varphi }\)=1 if \({WS}_{\varphi }\) is present in the \({n2}^{th}\) \(\mathrm{WFBN}\) belonging to class \(\vartheta ot\) otherwise \({\upbeta }_{n2, \vartheta ot}^{\varphi }\)= 0. \(\eta\) represents a small value to avoid the denominator being zero. After getting the ratio score for each frequent WEOS, arrange them in the descending order of the score.

2.4.3 Framework construction

The WEOS-based framework provides the basis for feature extraction. The discriminative WEOSs from both categories of subjects are used to construct the framework for feature extraction. The total of \(Q\) discriminative WEOSs is selected based on their ratio score. Half of these \(Q\) WEOS corresponds to the \(NO\_Task\) EEG data, and the remaining belong to the \(SIMKAP\_Task\) EEG data, arranged in a single row. There is no standard value available for the \(Q\); here, the value for \(Q\) is chosen based on the trial-and-error approach. Figure 2. represents the structure of the framework constructed.

Fig. 2
figure 2

Anatomy of the WEOS based framework

Full size image

2.5 Feature vector construction

The feature vector for classifying the different categories of subjects is obtained by comparing the WEOSs from both the train and test data with the discriminative WEOSs from the framework. If the WEOS from the \({j}^{th}\) subject is matched with the \({i}^{th}\) discriminative WEOS of the framework, then the \({i}^{th}\) column of the feature vector corresponding to the \({j}^{th}\) row will contain value 1 else 0.

The feature vector is obtained corresponding to each of the 4 subgraphs (i.e., DFS, BFS, DSP, MST) constructed from the WFBN. The feature vectors corresponding to each subgraph are further merged in different combinations in order to integrate the information from different subgraphs.

Figure 3 presents the structure of DFS subgraph of the WBFN constructed from 30 s segment without overlapping. In this figure, A and B represents the most discriminative WEOS corresponding to the SIMKAP_TASK and NO_TASK EEG, respectively. The C and D represents the samples belonging to the SIMKAP_TASK and NO_TASK EEG, respectively.

Fig. 3
figure 3

Structure of feature vector for DFS subgraph

Full size image

In this way, there are total 15 feature vectors (F1 to F15) are generated corresponding to each segment of EEG record (i.e., F1: DFS, F2: BFS, F3: DSP, F4: MST, F5: DFS + BFS, F6: DFS + DSP, F7: DFS + MST, F8: BFS + DSP, F9: BFS + MST, F10: DSP + MST, F11: DFS + BFS + DSP, F12: DFS + BFS + MST, F13: DFS + DSP + MST, F14: BFS + DSP + MST, F15: DFS + BFS + DSP + MST).

2.6 Classification and performance measures

The WEOS-based feature vectors obtained in Sect. 2.5 are classified using various algorithms. The classification algorithms employed in this study include K-nearest neighbors (KNN) [19], support vector machine (SVM) with linear, radial basis function (RBF), and polynomial kernels[20,21,22], and random forest classifiers [23]. The classification is performed using the ninefold cross-validation approach, and the final performance of the classification task is the average performance of all the folds.

The classification performance is quantified by calculating the performance measures from the confusion matrix generated after `classification. Here, four performance parameters are obtained corresponding to each confusion matrix: accuracy [24, 25], precision [26], recall [27], and Kohen’s kappa [28].

3 Results

This section provides the quantitative and structural analysis of the proposed approach for the mental workload estimation.

3.1 Quantitative evaluation

The quantitative evaluation presents the classification performance in terms of performance parameters depicted using a graphical representation.

Figure 4 presents the 9-fold classification accuracy for each of the 15 features corresponding to each 30 s long without overlapping segments of the EEG signal. The classification performance presented corresponds to each classification algorithm used in this study. In the figure, the letter S1-S5 corresponds to the five segments of the EEG signal, and F1-F15 corresponds to the fifteen features mentioned in Sect. 2.5. The figure depicts that the maximum classification accuracy of 96.67% is achieved using F12 features from S3 and S5 segments with the SVM (Linear) classification algorithm.

Fig. 4
figure 4

Segment-wise ninefold classification accuracy for EEG segment of length 30 s without overlapping

Full size image

Figure 5 depicts the ninefold classification accuracy for the WEOS-based features extracted from 50-s-long segment without overlapping. The figure presents segment-wise performance obtained using classification algorithms used in the study. The figure shows that the maximum classification performance with 98.89% accuracy is achieved using the F15 feature obtained from segment number S1 classified using the RF classifier. The same performance is also achieved using the F13 feature from S1 segment and SVM (RBF) algorithm.

Fig. 5
figure 5

Segment wise ninefold classification accuracy for EEG segment of length 50 s without overlapping

Full size image

In addition to the individual performance of each feature vector corresponding to each classifier, this study also investigates the average performance of each feature for all the classifiers used in the study.

Figure 6a and b presents the average performance of all the feature vectors, corresponding to all the segments of 30 s without overlapping and 50 s without overlapping segmentation, respectively, over all the mentioned classifiers. From Fig. 6a, it is observed that the feature vector F12 corresponding to the segment S5 of 30 s long without overlapping segments achieves maximum performance with 95% average accuracy.

Fig. 6
figure 6

Average accuracy for each feature vector from a 30 s segment without overlapping, b 50 s segment without overlapping

Full size image

Similarly, Fig. 6b, illustrates that the maximum average accuracy of 96% is obtained from feature vector F15 corresponding to the segment S1 of 50 s long without overlapping segments.

3.2 Structural analysis

The structural analysis of the proposed approach aims to characterize the connectivity pattern of the WFBN corresponding to a particular category of subjects and differentiate it from the connectivity pattern of another category of subjects.

In the study, this goal is achieved with the help of discriminative WEOSs. As discussed earlier in Sect. 2.4.2, the discriminative WEOSs represent the most frequent WEOS available in the WFBN corresponding to a particular category of subjects and are not available in the WFBN corresponding to the other category of subjects.

Figure 7a–d depict the dynamic connectivity of the discriminative WEOSs corresponding to the NO_TASK EEG data. Figure 8a–d depict the dynamic connectivity of the discriminative WEOSs corresponding to the SIMKAP activity EEG data. From Fig. 8a–e, it can be observed that, for the NO_TASK-related data, the brain's frontal region shows more active connectivity than the other regions. For the SIMKAP_TASK activity-related EEG data, the active connections span more area than that of the NO_TASK-related EEG data.

Fig. 7
figure 7

Discriminative WOS extracted from DFS subgraph corresponding to a Segment 1 b Segment 2 c Segment 3 d Segment 4 of NO_TASK EEG obtained using 30 s long segment without overlapping segmentation approach

Full size image
Fig. 8
figure 8

Discriminative WOS extracted from DFS subgraph corresponding to a Segment 1 b Segment 2 c Segment 3 d Segment 4 of SIMKAP_TASK EEG obtained using 30 s long segment without overlapping segmentation approach

Full size image

Hence, the structural analysis of the proposed methodology provides insight into the dynamics of brain connectivity corresponding to the NO_TASK and SIMKAP_TASK related data using discriminative WEOSs. The changes in the connectivity with respect to time help monitor a person's mental workload, which ultimately helps to manage the working environment for more productive and safe outcomes.

3.3 Discussion

Various approaches for EEG-based automated mental workload estimation are available in the literature. The comparative analysis of the proposed technique with state-of-the-art studies on the EEG-based mental workload estimation is presented in Table 1.

Table 1 Comparative analysis of proposed technique
Full size table

 

The first study [1] in Table 1 presented the EEG-based automated mental workload estimation. After feature selection, the most significant features obtained from EEG data are classified using a deep bidirectional long short-term memory—long short-term memory (BLSTM-LSTM) deep learning model. The study achieved a maximum classification accuracy of 85.33%. The second study [16] in Table 1, described an open access EEG dataset and presented an automated approach for mental workload estimation. This study considered three levels of workload conditions based on the ratings given by the subjects. The Power spectral density-based features are extracted from the delta, theta, alpha, and beta band of the EEG signal. Finally, the classification is performed with the support vector regressor, and achieved maximum classification accuracy of 69%. The third study [29] in Table 1 presented a restricted Boltzmann machines-based deep learning model for the EEG-based automated mental workload analysis. The study also developed an EEG channel selection approach to reduce the computational complexity. This study achieves a maximum classification performance of 96.10% accuracy.

The fourth study [30] presented a deep learning-based approach for drivers' mental workload analysis. The study achieved maximum classification performance with 92.68% accuracy. The fifth study [31] presents a deep learning model for engagement assessment with limited label information. The EEG data is recorded using the 4-h flight simulation, and the power spectral features are extracted. The presented approach achieves maximum classification performance with 86.52% classification accuracy. The sixth study [32] proposed an EEG-based approach for operator functional states classification, using the switching deep belief network with adaptive weights. This approach is common for the analysis of mental workload. The maximum classification performance with 76% classification accuracy is achieved with this technique.

The seventh study [33] proposed an improved Convolutional Neural Network (CNN) model for mental workload classification. The proposed technique considers EEG data's temporal, spectral and spatial information for feature extraction. The maximum classification performance with 92.37% accuracy is achieved using the presented method. The eighth study [34], suggested a deep recurrent neural network (RNN) based model for cross-task cognitive workload estimation. The maximum classification performance with 92.80% accuracy is achieved with the presented approach.

The ninth row of Table 1 represents the proposed study which outperforms all the state-of-the-art studies mentioned in the table. The proposed study presents the dynamic approach for mental workload estimation using the FBN. The proposed study achieves maximum classification performance with 98.89% accuracy using traditional classification algorithms. The important reason behind the achieved performance is nothing but the ability of the proposed network identifier to consider the weight information of connections and their ordinal relation to characterizing the FBN.

The study also provides the structural analysis of the proposed approach which enables researchers and neurologists to identify the affected brain regions due to the disability or the disorder. Here, structural analysis is performed using the discriminative WEOS, which represents the brain regions showing difference in connectivity for the different category of FBN.

4 Conclusion

The proposed study presented an advanced dynamic mental workload analysis approach using the WFBN approach. The WEOS-based identifier has proven to be effective for the EEG-based mental workload data with the significant performance achieved. The maximum performance with 98.89% classification accuracy is achieved using the RF classifier and F15 (i.e., DFS + BFS + DSP + MST) feature vector.

In addition to the quantitative analysis, the structural analysis provided a significant variation in the discriminative WEOSs-based connectivity patterns corresponding to the NO_TASK and SIMKAP_TASK related EEG data. The connectivity for the SIMKAP_TASK related EEG spans over the whole brain region, whereas the connectivity for NO_TASK related EEG is limited to the frontal region.

In the future, the study can be further extended to evaluate the various other application fields demanding brain functionality analysis.