Introduction

Motor imagery (MI) spans the action process and directly reaches the mental goal and sensory experience (Pilgramm et al. 2016). MI is a high-level cognitive process that has been widely applied to brain-computer inference (BCI) and motor recovery (Gao et al. 2021). Specially, MI-based BCI systems can help not only disabled people recover their lost physical functions (Khan et al. 2020) but also have innovative applications in robotics (Khademi et al. 2023), games (Yang et al. 2018), virtual reality (Kohli et al. 2022), and so on. Understanding the neural mechanisms underlying MI is crucial for optimizing BCI systems and enhancing our comprehension of motor-related cognitive processes. While previous studies have emphasized the conscious activation of specific brain regions during MI, the dynamic organization of brain states over time and the distinctions in activity between different MI tasks (i.e., left- and right-hand MI) remain unclear.

Numerous studies based on functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) neuroimaging techniques have investigated brain activation and network connectivity patterns during various MI tasks (Lebon et al. 2018; Li et al. 2019). For example, Hétu et al. (2013) applied the ALE meta-analysis to review the consistent activation of brain areas involved in MI, such as large frontoparietal network regions. In our previous studies (Zhang et al. 2016, 2019), we also found that functional connectivity patterns in the attention network and somatomotor network are associated with the context-specific MI task and MI-BCI performance. Bencivenga et al. (2021) evaluated the dynamic coupling of key regions using dynamic causal modeling and parametrical empirical Bayes analysis, revealing the interaction relationship between different regions in MI. Liu et al. (2022) constructed the causal connectivity network for left- and right-hand MI tasks using the Bayesian estimation, showing better lateralization characteristics of the brain network. Additionally, network properties of functional connectivity (Yu et al. 2022), event-related potential analysis (Daeglau et al. 2020), common spatial patterns (Zarubin et al. 2020), and other methods are extensively used to decode MI tasks. These techniques provide an activation or connectivity representation of the overall time series of MI.

However, MI is a dynamic evolutionary process, how the above features are organized in time and what kinds of activation and connectivity characteristics are possessed by different cognitive stages still need more dynamic research. Recently, more and more research has focused on the dynamics process of brain state under cognitive tasks through some prevalent methods. Adaptive directed transfer function-based (ADTF) algorithm and sliding window are two commonly used methods. To be specific, Zhang et al. (2018) explored the mechanism of the MI time-varying network using the ADTF method, they found the key role of some specific regions (such as the left anterior insula and premotor cortex) in the organization of the networks in MI. While, the sliding window method is widely used in the dynamic decoding of biosignal data in cognitive tasks due to its ability to dynamically extract a variety of neural signal features such as common spatial pattern (Talukdar et al. 2020), entropy (Parbat and Chakraborty 2021), and functional connectivity (Wu et al. 2021). However, the traditional ADTF method is easily affected by outliers and leads to the deviation of the established connection while Li et al. (2022) proposed a novel ADTF method to improve this defect. The sliding window approach also encounters the problem of decision window length and step size (Li et al. 2018), what’s more, the form of dynamic capture with a fixed window length is incongruent with the nature of the neural oscillatory activity (Quinn et al. 2018).

These studies based on biological imaging techniques have confirmed that MI is a high-level cognitive process involving multiple brain regions. Despite the variable factors, such as the difference in specific tasks, the signal acquisition methods, and the individual ability of the subjects, the activation of brain areas in the group-level study was still significant (Hétu et al. 2013). For more detail, whether network hierarchies can be found in higher cognitive tasks such as MI and how these networks are recruited in time is not known. To implement the MI-BCI system, it is also crucial to distinguish between left- and right-hand imagery tasks. Searching for potential pattern differences in MI of different hands in deeper temporal features is one of the aims of this study.

Alternatively, the hidden Markov model (HMM) provides a better choice. It is a generative model widely used in biological time series analysis which deduces the time series into the occupancy of finite states at discrete time points (Zhang et al. 2021). The inference of the model is completed in a data-driven way, which ensures that it is more flexible than the sliding window method to capture the dynamic information in the task (Hindriks et al. 2016). Based on these advantages, HMM is welcome in solving practical problems in cognitive neuroscience fields. For instance, HMM can provide information about the location of some brain-related diseases (Seedat et al. 2023), reveal the neural oscillation process of the human brain in cognitive tasks such as problem-solving (Yu et al. 2023), and extract the brain network features of the disease population to do classification research (Maya-Piedrahita et al. 2022). Specifically, the inherent advantages of HMM help to understand the dynamic transition process of brain networks in tasks. Moreover, HMM states, one of the keystones of the model. In addition to providing statistics of temporal features such as fractional occupancy and life time of the network (HMM state), task-containing HMM modeling can also discover those potential task-evoked state responses (Quinn et al. 2018). This task-dependent feature rises or falls is similar to event-related desynchronization/synchronization studies (ERD/S) (Vidaurre et al. 2016). At the same time, HMM could reveal neural oscillatory activity under specific tasks based on temporal organization and spatial topology. These simplify the dynamic recruitment system of brain networks and allows us to glimpse into the mysteries of human cognition. Thus, the HMM method provides a new way to insight into the neuromechanism of MI.

We followed a dual-task modeling strategy to search for potential differences in the temporal organization of left- and right-hand MI under large-scale network dynamics. This modeling design allows us to discuss pattern differences under the same state assumptions. A state transition system was revealed by modeling the HMM on the concatenated MI-EEG data of left and right MI. In this study, we first perform HMM modeling on EEG-MI data concatenated from left- and right-hand tasks. Then, the overall mode of hand MI task is discussed based on the three temporal features of fractional occupancy, life time and interval time, and the model feature of the transition probability matrix. In addition, studies of task-evoked responses revealed large-scale state recruitment patterns for those that are strongly task-relevant. Furthermore, we investigated the differences in the above features between the left- and right-hand MI tasks by paired two-sample t-tests. Finally, the correlations between these features and BCI performance were examined by Pearson correlation analysis. This comprehensive exploration, facilitated by the Bayes Hidden Markov model, provides valuable insights into the brain states and dynamic transitions underlying MI, advancing our understanding of neural mechanisms during motor tasks.

Materials and methods

Participants

This study utilized the freely available MI-EEG dataset (Goldberger et al. 2000), which can be downloaded at https://physionet.org. In this MI-BCI experiment, a total of 109 subjects performed various motor execution/MI tasks. The EEG dataset was recorded by the BCI2000 system (10–10 international system) with a 64-channel and a sampling rate of 160 Hz. This paper aims to investigate the dynamic changes in brain states under hand MI task. Only data from the left hand and right hand when imagining opening and closing the fist are used here. After EEG data preprocessing, 43 subjects were finally selected for the current study. Due to the use of a publicly accessible database, no ethical approval was provided for our current study on human participants.

MI-BCI experiment procedure

The MI-BCI experiment consisted of 14 runs, each corresponding to a task or baseline run. Since this study focused on single-hand MI tasks, only runs with left- or right-hand performing MI tasks alone were extracted for subsequent analysis. A total of three runs contains the tasks we focus on and each run contains 15 trials. Each trial consisted of 4.1 s MI task time and a rest interval of 4 s. Here we extract 1 s before task onset and 4 s after task onset total of 5 s as one epoch for further research. The composition of an example trial is displayed in Fig. 1. The task instructions were provided to the participants at 0 s, and a cue target would appear on the screen's left or right side. The subjects were instructed to imagine their hands closing and opening in the corresponding direction to the target's direction. Refer to (Schalk et al. 2004) for additional information.

Fig. 1
figure 1

The schematic diagram of the experimental paradigm

Full size image

EEG data preprocessing

All EEG data were preprocessed using a traditional analysis procedure. The main analysis steps are as follows: (1) 8–30 Hz (including mu rhythms and beta rhythm) band-pass filtering on the raw EEG datasets (Fadel et al., 2020). (2) Eye movement artifacts removal by the ICA method. (3) Artifact trail elimination (± 100 μV as the threshold). (4) [− 200 ms, 0 ms] baseline correlation was done to compare the effects of stimulating events on brain activity. After that, there were 69 subjects left. (5) Subjects with less than ten trials were removed to minimize the effect of chance. Therefore, another 23 participants were excluded. (6) Deleted superfluous eight channels (AF7, AF8, FT7, FT8, T9, T10, Iz, Oz) according to a previous study (Kang et al. 2018). (7) Three subjects without information on MI-BCI performance based on Kang’s study (Kang et al. 2021) were deleted. Finally, only 43 subjects were left after preprocessing for further analysis. The workflow of MI-EEG is shown in Fig. 2, including data preprocessing, HMM inference, and statistical analysis. The specifics of these steps are described in the following sections.

Fig. 2
figure 2

The workflow of brain state and dynamic transition patterns of the left- and right-hand MI using the HMM method. a Data preprocessing and trials concatenate. b HMM inference on MI-EEG data. c Statistical analysis of HMM features across tasks and correlation analysis of HMM features with BCI performance

Full size image

Gaussian-HMM inference

The HMM with Gaussian observation describes the dynamic activity of the brain as the transition of limited states, and states are defined by functional connectivity and mean activation between regions (Javaheripour et al. 2023). HMM states can be thought as \(k\) multivariate Gaussian distributions, and they can be described by the following mathematical expression, which \(k\) stands for the number of states.

$$P(X_{t} |S_{t} = k) = P(X_{t} |\mu_{k} ,\sum\nolimits_{k} {} )$$
(1)

While \(\mu_{{\text{k}}}\) represents the mean activation, \(\sum\nolimits_{{\text{k}}} {}\) represents the functional connectivity across regions of each state (defined by the covariance matrix). The switching of states over time defined by networks and activations provides a dynamic perspective to understand the temporal organization of brain activity.

The HMM inference method used in this paper follows the approach previously described by Vidaurre et al. (2017) and the toolbox can be obtained through MATLAB toolbox HMM-MAR (https://github.com/OHBA-analysis/HMM-MAR). This toolbox provides a variational Bayes framework to infer the HMM which is also guaranteed to converge as the popular expectation maximization (EM) algorithm. The keystone of this method is to find the approximate solution to the posterior distribution by minimizing the cost function. Here, Kullback–Leibler divergence (KL divergence), as a commonly used metric to measure the distance between two distributions, is adopted as the cost function. For observation data \(X = \left( {X_{1} ,X_{2} ,...,X_{T} } \right)\), HMM needs to infer the hidden state parameter \(S = (S_{1} ,S_{2} ,...,S_{T} )\) and model parameter \(\lambda = (\pi ,A,B)\), where the model parameters include three parts: (1) initial state probability \(\pi\), (2) state transition probability \(A\)(\(A\, = \,\left\{ {a_{ij} } \right\}\), where \(a_{ij}\) denotes the transition probability from state \(i\) to state \(j\)), and (3) observation probability \(B\)(which is also \(P(X_{t} |S_{t} )\)). Therefore, the minimization function can be expressed as follows:

$$F = KL(Q(S)Q(\lambda )||P(S,\lambda ,X)) = \iint {Q(S)Q(\pi )Q(A)Q(B)}\log \frac{Q(S)Q(\pi )Q(A)Q(B)}{{P(S,\pi ,A,B,X)}}d\lambda dS$$
(2)

Here, \(P(S,\lambda ,X)\) is the full true posterior distribution that needs to be approximated. The optimization iteration is performed by minimizing \(F\) with respect to \(Q(S)\), \(Q(\pi )\), \(Q(A)\), and \(Q(B)\) individually. Additionally, it is also important to mention that all parameters are given a prior distribution to start the optimization process. The Dirichlet distribution is a common prior choice for \(Q(\pi )\) and \(Q(A)\) while the conjugate priors to \(\mu_{k}\) and independent inverse Gamma prior to \(\sum\nolimits_{k} {}\) (Tao et al. 2021). The parameter update process is repeated until the change in \(F\) is less than a given threshold. At this time, it is considered that the KL divergence no longer has a significant change, and the estimated parameters can be output. All the basic parameters of HMM can be obtained based on the above inference process.

State selection and model parameters of HMM

Brain states play an extremely important role in HMM, and all model features are discussed based on state. However, the choice of the number of HMM states has always been a controversial topic because there is no systematic method to solve this problem. Common strategies such as using free energy to quantitatively select the number of states also face situations where there is no turning point (Lin et al. 2022), a lower free energy represents a better fit of the model to the data (Zarghami and Friston 2020). It should be noted that HMM states do not fully represent biologically factual processes, different state number settings reveal brain dynamics from different perspectives (Vidaurre et al. 2018b). In this study, the number of HMM states and the number of Principal Component Analysis (PCA) components are considered comprehensively, and the two parameters are determined by a classification accuracy-oriented strategy.

Principal component analysis (PCA)

Due to the multi-channel signal properties of EEG, more signal channels mean more redundant information. To alleviate this problem, PCA is a very general choice. The PCA method can reduce the dimension of the multi-channel data so that the model inference can work in the PCA-reduced space. The HMM inference toolbox used in this paper contains the option of PCA analysis, which only needs to be manually set for a given number of PCA components.

A classification accuracy-oriented model parameter selection strategy

We adopted a classification accuracy-oriented model parameter selection strategy, and combining the outcomes of PCA-explained variance allowed for a final determination of the number of HMM states and PCA components. The entire parameter selection processes are as follows: first, given a range of 3–12 for the number of states and 2–56 for the number of PCA components. The model is then trained for all possible combinations. For each model, the naïve kernel of the state probability was extracted as the feature vector (Ahrends and Vidaurre 2023). We first estimate a state-space model of brain dynamics at the group level. Then, dual estimation was used to calculate the subject-specific model features. Finally, the dynamic features of each subject are obtained by projecting the subject-specific model parameters into the embedding space using the naïve kernel. After obtaining the dynamic characteristics of the subjects through the above methods, the Support vector machine (SVM) with a linear kernel was used for classification. A leave-one-out cross-validation strategy was adopted to evaluate the performance of the classifier. The top 10 combinations of classification accuracy are shown in Supplementary Table S1.

Although the combination of K11 and PCA2 had the highest accuracy, we adopted the combination with the second highest accuracy as the final parameter for the explained variance of PCA number 2 was too low. The classification accuracy corresponding to different parameter combinations and the corresponding explained variances for different PCA components are shown in Fig. 3. Therefore, we finally determined an HMM with 11 states and a PCA component parameter set to 4 for subsequent analysis.

Fig. 3
figure 3

Selection of k value and PCA components. a Accuracy corresponds to different parameter combinations. b Explained variance for different numbers of PCA components

Full size image

Statistic overview of HMM

To comprehensively understand the time and transition features of the MI task, we used the Viterbi path (Lember et al. 2019) and Gamma (Van Schependom et al. 2019) inferred by HMM to conduct subsequent feature calculations. Viterbi path is a hidden state time series sequence estimated by the Balm-Welch algorithm (Rezek and Roberts 2005). Its state assignment at each time point is completely exclusive (Bishop and Nasrabadi 2006). The opposite of the Viterbi path is Gamma, which is the posterior distribution of the time series presented in probabilistic form. Its state assignment at each time point is not exclusive and more susceptible to dynamic change (Quinn et al. 2018).

In this paper, we mainly consider the following features: fractional occupancy, maximum fractional occupancy, transition probability matrix, mean life time, and mean interval time. Notably, \(S_{t} = = k\) represents the logical judgment (returns 1 if correct or 0 if wrong).

  1. (1)

    Fractional occupancy (FO):

    FO of state represents the proportion of time that a state is active to the total time.

    $$FO(k) = \frac{1}{T}\sum\nolimits_{t} {(S_{t} = = k) \times 100\% }$$
    (3)
  2. (2)

    Maximum fractional occupancy (Max FO):

    Max FO represents the highest visit ratio among all states of a subject.

    $$MaxFO = Max(FO(k)) \, (k = 1,2,3,4,5,6,7,8,9,10,11)$$
    (4)
  3. (3)

    Transition probability matrix (TPM):

    The TPM represents the transition probability from one state to another.

    $$TPM = \left\{ {a_{ij} } \right\}{ (}i,j = 1,2,3,4,5,6,7,8,9,10,11 \, i \ne j)$$
    (5)
  4. (4)

    Mean life time (LT):

    The mean LT represents the mean duration of a state's continuous persistence.

    $$mean \, LT(k) = average(\sum\nolimits_{k} {(S_{t} = = k)} ) \, (k = 1,2,3,4,5,6,7,8,9,10,11)$$
    (6)
  5. (5)

    Mean interval time (IT):

    The mean IT is the average number of time points between two state occurrences.

    $$\, mean \, IT(k) = average(\sum\nolimits_{t} {(((S_{t} = = k) - (S_{t - 1} = = k))} = = 1) \, (k = 1,2,3,4,5,6,7,8,9,10,11)$$
    (7)

ERD/S analysis

ERD/S patterns reveal the time-locked neural activity responses evoked by specific task stimuli (Pfurtscheller and Neuper 2006). As it has been widely used for pattern decoding of MI-EEG data in the past, ERD/S responses were also calculated to compare with task-evoked responses revealed by hidden Markov models. The calculation of ERD/S time course was performed in the 8–30 Hz frequency band which is consistent with the data frequency band in HMM inference. The whole calculation procedure included: (1) The Hilbert transform was applied to the signals of C3, Cz, and C4 during left- and right-hand MI tasks across all trials and subjects. (2) Then, the absolute values were taken for each complex value of all trials. (3) Following the formula \(ERD\% = \frac{U - R}{R} \times 100\%\) to calculate the relative power refers to previous research (Cho et al., 2017). Here, \(U\) represents the time series of each trial period (−1 to 4 s) while \(R\) representing the mean power of the baseline period (−1 to 0 s).

BCI performance

BCI performance can be used as a potential behavioral indicator, and we can investigate which dynamic features are related to the BCI performance value. To find those underlying dynamic features that are easier to distinguish the left- and right-hand MI tasks. Thanks to the results of Kang's BCI illiteracy-related study (Kang et al. 2021), the BCI performance values of subjects in the BCI2000 dataset were provided (taking the classification accuracy as the BCI performance value of the subjects). Here, we adopted the average of the classification accuracy under three deep learning methods as BCI performance. All the BCI performance values are shown in Supplementary Table S2. And all the HMM temporal features used for BCI correlation analysis were averaged over the left- and right-hand tasks.

Statistical analysis

To compare the state activation probabilities evoked by the MI task with those in the idle period, we performed paired t-tests on each task-evoked time point, where the significant difference level was set at p < 0.05 (FDR corrected, p < 0.05). To compare the temporal feature differences between the left- and right-hand MI tasks, we also performed paired t-tests, where the significant difference level was set at p < 0.05 (uncorrected) for FO, LT and IT. For TPM, the significant difference level was set at p < 0.01 (uncorrected). We used Pearson’s correlation analysis to evaluate the relationships between the MI-related HMM state temporal features and BCI performance, where the significant correlation was set at p < 0.05. It is worth noting that the BCI performance represents the individual overall ability to identify left- and right-hand MI, so the sum of MI-related HMM state temporal features (FO, LT, and IT) of left- and right-hand MI was used to calculate the correlation.

Results

HMM state patterns of MI tasks

The HMM states were estimated at the group level that concatenated all MI (left- and right-hand) trials and all subjects. Here, eleven HMM brain states were inferred based on the MI-relevant task EEG data. Figure 4a shows an example of the MI task state time series in a 1-s section for one subject. These HMM states describe the unique brain activity and functional connectivity patterns across time, where the activation maps of four states and corresponding functional connectivity matrixes were displayed.

Fig. 4
figure 4

Temporal features identified by Gaussian-HMM on MI-EEG data. a State-dependent time series for one example subject where each state represents a brain network with variant activation pattern. b The maximum fractional occupancy distribution of the subjects. c Transition probability matrix (represents the transition probability from one state to another). d The distribution of fractional occupancy (represents the proportion of time that a state is active to the total time) and life time (defined as the duration of each state occurrence) of each state. e The distribution of interval time (represents the interval between two visits of each state)

Full size image

Moreover, four basic overviews of the brain HMM states during the MI task are shown in Fig. 4b–e. First, the maxFO measures whether an HMM is able to describe the dynamics of the data well. In the present model, we found that the distribution of maxFO of the subjects is mainly concentrated between 10 and 26% (see Fig. 4b). This indicates that the HMM successfully describes the dynamics of the MI-related data. We also assessed the state transition probability matrix, which represents the transition probability from one state to another state. According to Fig. 4c, we observed that some transitions (i.e., S8–S11, S2–S9, and S5–S8) are more probable than others. Figure 4d shows the LT and FO for each HMM state. The FO of the eleven states ranges from 6.84 to 14.02% while the LT ranges from 29.3 to 52.1 ms. We found that S11 exhibited a relatively higher occupancy, whereas S10 was less frequent. Figure 4e shows the interval time for each state, varying from 199.3 to 591.5 ms. Interestingly, the S10 occurs infrequently, but once it occurs it persists for a long time. On the contrary, S11 has the shortest interval, revealing more frequent access to that state.

Metastate identification and MI-task evoked response

Similar to resting-state brain activity, task-evoked brain activity is also highly organized. Three metastates were identified by performing ward clustering analysis on the correlation matrix of FO. Figure 5a shows the correlation matrix of FO and the graph of hierarchical clustering. The colored boxes represent the correlation of state occupancy showing a higher positive correlation within metastates and lower correlations between metastates. Instead, we find low transition probabilities within metastates and high transitions between metastates (Fig. 5b).

Fig. 5
figure 5

State clustering results and task-evoked response. a FO correlation matrix, exhibits positive correlation within metastate across subjects. b TPM, the transition probability within a metastate is lower than the transition probability between metastates. c Task-evoked response, measured by the mean probability of state activation across trials along with the thick red line which represents the mean ERD/S in electrodes C3, Cz, and C4. d The spatial cortical distribution corresponds to the eleven Hidden Markov states

Full size image

Since the inference of the HMM does not leak information about the task onset time, we can find the task-evoked responses from the inferred state time series and its features. Figure 5c shows the probability of activation of eleven HMM states during MI across time. The value of each point represents the proportion of state activation across trials, subjects, and MI tasks. To compare the state activation probabilities evoked by the MI task with those in the idle period, we performed paired t-tests on each task-evoked time point. We averaged the state activation probabilities at all time points within 1 s before the MI-task onset and took the average as the comparison baseline. We found that the S1–3, S5, S7–8, and S10–11 have significant differences (FDR correction, p < 0.05) in activation probability. In order to better observe the MI task-evoked state activation response, we divided 4 stages according to the interval of 1 s. We found that S1 and S5 were significantly different from their corresponding idle period across stages, S10 and S11 were significantly different from their corresponding idle period in stages 1 and 2, and S2, S3, and S7 were significantly different from their corresponding idle period in stage 1. Moreover, we calculate the mean ERD/S of electrodes C3, Cz, and C4 across all trials and subjects. We found that the probability of activation for some HMM states have a similar trend to that of the ERD time course of MI. Figure 5d shows the map of state activation. Different states correspond to different activation patterns or network connections.

Differences in temporal features of HMM states between left- and right-hand MI

Since we concatenate the left- and the right-hand MI data to identify the group HMM states, that allows us to compare the temporal feature differences between the two MI tasks based on the same state assumption. Given the MI task-evoked response (see Fig. 5c), we compared the FO, mean LT, mean IT, (p < 0.05) and TPM (p < 0.01) differences between the two MI tasks across stages and states. Figure 6a shows the difference in FO. In stage 1, we found that the FO of S1 and S7 in left-hand MI were significantly lower than right-hand MI, indicating S1 (right occipital lobe) and S7 (right parietal-occipital lobe) have longer access (dwell) time for right-hand MI. In stage 2, we found that the FO of S4 in left-hand MI was significantly higher than in right-hand MI. In stage 4, we found that the FO of S2 in left-hand MI was significantly higher than in right-hand MI, while the opposite situation was observed for S6. In all-stage, we found that the FO of S7 in left-hand MI was significantly lower than in right-hand MI.

Fig. 6
figure 6

Statistical overview of differences between left- and right-hand MI tasks in different stages. a The differences in FO. b The differences in mean LT. c The differences in mean IT. d The T-value matrix of the difference between the left–right hand TPM. The asterisks represent p-values less than 0.05, double asterisks represent p-values less than 0.01 and triple asterisks represent p-values less than 0.005. LH-MI: left-hand MI; RH-MI: right-hand MI

Full size image

For the mean LT, we found significant differences in some states except stage 4. We found the LT of S4 in left-hand MI was significantly higher than right-hand MI in stage1–3 and all. Compared to right-hand MI, the LT for S1 in stage 1 and S6 in stage 3 were significantly lower in the left-hand MI, while the opposite situation was found in S10 during stage 2, S3 during stage 3, and S11 during all-stage. For the mean IT, compared to right-hand MI, we found that the IT of S5 in stage 1, S7 in stage 4, and S10 in all-stage were significantly lower in the left-hand MI, while the opposite situation was found in S8 during stage 3. Moreover, there are also differences between left- and right-hand MI in some transitions in stage 1, stage 4, and all-stage as shown in Fig. 6d.

The correlations of HMM state temporal features and BCI performance

Figure 7 shows the significant correlations between BCI performance and temporal features of HMM states. Since we only obtained the average classification accuracy of left- and right-hand MI, we only focused on the relationships between MI tasks and BCI performance here. For all temporal metrics, the average of left- and right-hand MI was used to represent the temporal features of the hand MI. We found that the FO of S1, S3, and S4 were significantly negatively correlated with BCI performance (r =  − 0.344, p = 0.024; r =  − 0.320, p = 0.037; r =  − 0.348, p = 0.022), while the FO of S10 was significantly positively correlated with BCI performance (r = 0.352, p = 0.021). Moreover, we found the LT of S10, the IT of S1 and S4 were significantly positively correlated with BCI performance (r = 0.403, p = 0.007; r = 0.303, p = 0.049; r = 0.309, p = 0.044), while the IT of S11 was significantly negatively correlated with BCI performance (r =  − 0.339, p = 0.026).

Fig. 7
figure 7

Correlations between HMM states’ temporal features and BCI performance. a The correlation between BCI performance and FO. b The correlation between BCI performance and LT. c The correlation between BCI performance and IT

Full size image

Discussion

MI, an extraordinary skill of the human brain to simulate movement, has attracted much attention for its extensive application prospects (Guillot et al., 2014). It is a classical cognitive task involving multiple psychological concepts such as attention, working memory, episodic memory, and so on (Guillot et al., 2014; Munzert et al. 2009). The brain's network temporal organization during cognitive tasks is as nonrandom as the resting state and the task-evoked state activity responds on millisecond timescales (Quinn et al. 2018). To understand the dynamic brain process that can maintain the intrinsic temporal benefit of EEG, HMM offers a high-temporal resolution state-switching hypothesis. In other words, the HMM method can reveal that the state-switching pattern does not lose the time information of EEG data like the sliding window method. Not only that, HMM also performs better in the discrimination of functional connectivity (Duc and Lee 2020). The analysis of such repeated access patterns not only uncovers the dynamic organization of the network for MI tasks but also provides a comparison of the temporal organization differences among different tasks. In the current study, we identified 11 recurrent dynamic brain states and revealed the MI process through the temporal features of the transition between them and the dynamic recruitment. Our findings uncovered dynamic patterns, activation differences, and temporal features, shedding light on the underlying neural dynamics.

The HMM provides a dynamic representation of the state transition as shown in Fig. 4a. It exhibits the speculative HMM state-switching process in the form of probabilities, which characterizes the time series in terms of transitions of finite states (Vidaurre et al. 2017). The state defined by a specific spatial topology and its temporal organization decodes the biological signal of the relevant task. MaxFO is a metric that reflects the time distribution of states. A lower MaxFO means that the time distribution of the state tends to be more average than concentrated in a particular state. In the model of this paper, the maxFO is concentrated between 0.1 and 0.26, indicating that the model captures the dynamics of the data well (a single state does not highly dominate the entire time series) (Vidaurre et al. 2018a). TPM provides a window into the interaction of the brain network under task (Hunyadi et al. 2019). The four pairs of transitions with the highest probability occur between S8–S11, S2–S9, S5–S8, and S6–S3. We found that these state transitions occur primarily from parietal to temporal, parietal to parietal, and frontal to frontoparietal regions. Notably, the frontoparietal regions along with the prefrontal regions are responsible for information integration in MI (Ogawa et al. 2022). We suggest that the brain is not entirely inclined to switch between two totally different states, whereas those higher transition probabilities are also found in similar states during MI. These state transitions can be regarded as an interim between networks or the partial maintenance of activation.

The temporal properties of each state are characterized by its FO, LT, and IT. The statistics of these temporal features can delineate the dynamic recruitment pattern of each state (Lin et al. 2022). Specifically, we found that the S10 (activation from frontoparietal to occipital) was the least occupied state, which may represent its transient involvement in the MI task. It is characterized by a high life time and high interval time, which suggests that this state is not recruited frequently but persists for a long time once recruited (see Fig. 4d and 4e). This state has the widest activation region of all the states. We speculate that the appearance of this state may be a rapid brain response to the task (S10 had the earliest increased task-evoked response, which indicates it involved in the initial stages of cognitive processes throughout the task), with multiple regions are mobilized for efficient execution of MI. The prefrontal cortex involves brain functions such as executive control and working memory, while the parietal cortex is involved in the selection of visual attention (Capotosto et al. 2013; Scolari et al. 2015). The occipital lobe is generally considered to be a region related to visual representation (Slotnick et al. 2012). The fast response as well as the activation of multiple regions may represent a rapid emergency response of the brain. The state-11 is a state of temporal lobe activation. Although the temporal lobe functions primarily in the auditory system, the temporal lobe may be involved in various ways in the MI. Several studies have demonstrated that the temporal lobe contains areas such as the superior temporal sulcus (STS) that respond to visual stimuli (Beauchamp 2015; Petrides 2023). In addition, activation of state-11 may also be involved in the overall MI process by encoding memories. It has been shown that specific temporal lobe regions (medial temporal lobes (MTL) which include the hippocampus) have memory-encoding structures (Pearson et al. 2015; Rolls et al. 2022). The recognition of MI task cueing information can be viewed as a top-down perceptual process, as the signals only cue the direction and start time of the task but do not display imagery content. Specifically, subjects must generate corresponding mental imagery by searching for their prior motor experiences (Mulder et al. 2004; Olsson and Nyberg 2010; Pearson 2019). Thus, these eleven distinct brain HMM states during MI tasks could offer a comprehensive view of the dynamic nature of MI brain activity.

Using the clustering analysis, we could further divide the eleven distinct brain states into three metastates. We observed the states within these three metastates had a strong FO correlation (Fig. 5a) (Vidaurre et al. 2017), and the transition probability within metastates was not as high as between metastates in terms of the TPM (Fig. 5b). This trend is particularly evident in metastate 1 and 2. Moreover, the spatial topology of the states in the two metastates is very similar. This may suggest that metastate 1 and 2 are functionally similar, and the cross-network interactions are frequent in the MI task. Additionally, it is equally feasible to analyze state reflections of task dependence at the group level (Quinn et al. 2018), although HMM is presented in mutually exclusive state recruitment patterns, and different subjects are not assumed to perform cognitive tasks simultaneously. We found contrasting state responses using cross-trial statistics of states at each time point, that is, state activation probabilities represented by the proportion of trials in that state at each time point (Quinn et al. 2018). These changes are found in some states in all three metastates. Here, we only focus on those states with increasing occupancy because they may be associated with the performance of MI tasks. Meanwhile, the states that maintain a higher activation probability than the idle period over multiple phases are also of interest. First, stage 1 can be regarded as a rapid response stage, which may include various functions such as receiving task stimuli and information integration. This phenomenon can be found not only in the task-evoked response of the HMM states but also in the ERD/S pattern of C3, Cz, and C4. Specifically, the ERD/S time series shows the magnitude oscillation at stage 1 and the oscillation trend is similar to the activation probability of some states (e.g. S1, S7, and S10). ERD/S is a method that quantifies event-related changes and is used to detect patterns of cortical activity in MI (Neuper et al., 2010). By contrast, the fluctuations in state occupancy over time provided by HMM are more informative, because such fluctuations are based not only on a single electrode but on an identified brain network. These two have similar trends in the form of fluctuations, indicating that the HMM brain state activation probability function can also well reflect the brain activation pattern of MI. Second, S10 is a key state showing a significant occupancy increase and both FO and LT of temporal features are positively correlated with individual’s BCI performance (see Fig. 7). The rapid response in this state reflects the large-scale and multiregional brain activation that may be evoked by the MI-task stimulus. Among them, the frontal cortex (including the motor cortex and premotor cortex, etc.) is related to behavioral organization (Decety 1996), and the increased activation of the frontal lobe may be related to attention or effort (Van der Lubbe et al. 2021). A previous MI study in elderly subjects has suggested that the occipital lobe may participate in MI tasks through visual imagery (Zapparoli et al. 2013), which is a mental technique for MI tasks. Similar results were also found in patients with parietal lesions when they performed the MI (Madan and Singhal 2012). Visual imagery has been used as an alternative strategy in MI tasks. At the same time, higher occupancy in S10 seems to predict a better individual’s BCI performance, which may be explained by the fact that the three BCI classification methods are more sensitive to the temporal characteristics of large-scale activation states. The following states with rising probabilities are S3 and S11. S3, with dominant activation in the prefrontal lobe, and S11 with primary activation in the temporal lobe, may represent different cognitive strategies representations. Compared to S10, their activation areas are concentrated and may represent specialized organizational functions. Imagining specific body movement processes can be seen as recalling episodic memory. Although S3 and S11 have different activation areas, they may have similar functions. The prefrontal lobe was found to be activated during the episodic task experiment (Nolde et al. 1998) and support the organization function of cognitive behavior (Wei and Luo 2010). In addition to auditory functions, the temporal lobe contains specific neural structures such as MTL and hippocampus (Kiernan 2012) that play a role in functions other than hearing. The hippocampus guides forward actions based on past experiences in an agile way (Vernon et al. 2015). In fact, these two states are not exclusive in their function. On the contrary, the functions of the Prefrontal–hippocampal cortex (PFC) and hippocampus in memory processing are complementary, with the former responsible for retrieving suitable memory content and the latter for organizing memory (Eichenbaum 2017). Finally, the higher activation probability of S1 from stage 2 to stage 4 may be related to different cognitive strategies in the pure imagination stage. It must be admitted that there are differences in the performance of imagination tasks, and consistent mental representations are almost non-existent (Milton et al. 2008). For example, previous study indicated that the older adults have more difficulty performing kinesthetic imagery tasks (Zapparoli et al. 2013), and instead, they need to recruit visual imagery related regions (which activates more occipital areas) (Fallgatter et al. 1997). Another observation is that S11 holds a higher activation probability at stage 2 compared to the corresponding idle period. At this stage, it may indicate some functions of planning and preparation (Burianová et al. 2013). This trend disappeared over time because successful memory encoding is often negatively correlated with activation in some temporal regions (hippocampus) (Confalonieri et al., 2012; Milton et al. 2008).

According to the preceding task activation results, the large-scale recruitment of states is revealed variously at different time stages. The differences in temporal features between left- and right-hand MI also differed within the time segments described above. It is worth noting that stage 1, as a stimulation-induced fast reaction period, shows the features of the most difference. Besides, the right-hand task showed a higher occupancy rate for the activation state (S1, S7) of the right occipital lobe in multiple stages, while the left-hand task had a higher occupancy rate for the activation state (S4) of the left occipital lobe, there was an ipsilateral preference in the occupancy of states dominated by occipital activation. However, the ipsilateral preference of left and right hands for occipital states did not appear simultaneously. Right-hand right occipital preference was significant in stage 1, while the left occipital preference in the left-hand task was significant in stage 2 (this occupancy preference exists both globally and on a single access). This difference may stem from the fact that the task processes of the left- and right-hand MI tasks are not synchronized. This is unsurprising because movement imagination is relatively slower in the non-dominant hand, and the difference is more pronounced than in motor execution (Maruff et al. 1999). These results indicated that the laterality of neural recruitment was evident, with left-hand MI showing distinct activation patterns in the occipital lobe, while right-hand MI exhibited a preference for the right hemisphere.

Another notable difference comes from S10 and S11. These two states were discussed earlier because of their unique temporal features, and they were also found in the difference between the left- and right-hand MI tasks. In stage 2, the S10’s LT of left-hand MI was longer. However, in all-stage, left-hand MI had shorter access intervals in the S10. One assumption for the difference in S10 at stage 2 is that, as we discussed earlier, S10 is the brain state that responds rapidly to simulation. This response will be longer for the non-dominant hand task because that kind of task usually takes up more mental resources (Tacchino et al. 2018). In addition, shorter intervals for S10 in left-hand MI also indicate more frequent visits in terms of all-stage. S11 was found to have higher single access times for left-hand MI tasks in both stage 3 and all-stage. Globally, the left-hand task showed longer single visits to temporal states (S11) and shorter intervals between visits to multi-activation states (S10) which represent a particular pattern of this task. Finally, some differences in left- and right-hand MI were found in the transition probability matrix. A notable characteristic of these differences is that the transitions with the activation state of the left central parietal region (S5, S8) as the transition endpoint during the period in which the differences emerged all had higher values in the right-hand MI. The transition to the state with the right central area activated seems more relevant to the right-hand MI. Past studies also support the conclusion that the left motor area is vital for the right-hand MI task (Gao et al. 2011). Overall, our findings provide valuable insights into the neural mechanisms underlying left- and right-MI tasks.

Regarding the limitations of this study, as with all imaging studies, inferring the internal processes of the brain from exogenous physiological measurements carries its own risks. Relying on temporal statistics as well as backward reasoning about the electrophysiological signals of the brain provided by the hypothetical model can only be considered as a conjecture. More evidence is still needed on the real activity of the brain under cognitive tasks in future studies. At the same time, it is limited by the low spatial resolution of EEG. The fuzzy localization of activation in specific states leads to an imprecise discussion of specific cortical functions. In the future, multimodal studies combining functional magnetic resonance imaging (fMRI) data would be a better choice. In addition, the single HMM brain state temporal features are associated with MI activation patterns, and future studies can explore integrating multiple temporal characteristics of these HMM brain states to further improve the accuracy of MI-BCI classification.

Conclusion

In this study, we delved into the dynamic organization of brain states during MI tasks using an HMM approach. Through the analysis of left- and right-hand MI-EEG signals, we identified eleven distinct states at the group level, revealing a highly organized structure in MI-related brain activity. The dynamic transitions between these states emphasized the fluid nature of task-evoked brain responses. Our findings highlighted specific states with unique activation patterns across different stages of MI tasks, shedding light on the temporal dynamics of MI-related neural activity. Notably, the comparison between left- and right-hand MI tasks unveiled significant temporal differences in FO, mean LT, mean IT, and transition probability matrix across stages and states. The left-hand MI task exhibited higher recruitment in the occipital activation state skewed to the left hemisphere, while the right-hand MI task showed a preference for the right hemisphere. Furthermore, the correlation analysis between HMM state temporal features and BCI performance provided valuable insights. Specific states, such as S1, S3, S4, and S10, exhibited significant correlations with BCI performance, with FO, LT, and IT playing distinct roles in predicting performance outcomes. In general, our findings support the idea that MI may be performed with different cognitive strategies and that these strategies are all feasible.