1 Introduction

The brain–computer interface (BCI) is an exciting topic of neuroscience, physics and engineering. Such modern technology is in demand in various applied fields, including medicine, industry and others [1,2,3,4,5,6]. The BCI is known to be based on real-time detection of characteristic forms of electrical (or magnetic) activity of the brain and the transformation of the obtained information into computer commands for controlling hardware. At present, the developed neurointerfaces allow one to control the 2-D movement of a cursor [7], partially synthesize speech [8] and control simplest movements [9]. The BCIs can be effectively used for rehabilitation [10], controlling exoskeletons [1] and robots [11].

The BCIs use both noninvasive or invasive methods for recording the brain electrical activity and then transform it into user’s control commands [12, 13]. It was initially assumed that only invasive BCIs, which use electrodes implanted in the brain, can provide multidimensional movement control of a robotic arm or a neuroprosthesis [14,15,16]. At the same time, noninvasive BCIs are shown to be prospective devices for basic communication and control [7, 17]. Noninvasive BCIs are based on the online analysis and extraction of the features of multichannel recordings of electrical brain activity (EEG), which is the powerful tool for study of the brain activity [18]. It allows to estimate the degree of the involvement of neural ensembles in generation of different rhythms and the interactions between neurons belonging to different brain structures [19, 20]. The BCI operation is largely determined by the possibility of generating human stable and reproducible patterns of cognitive activity, which then can be translated for control commands. In this context, the most promising approach is the use of imagination of motor activity [21].

There are plenty of techniques for analyzing neurophysiological features of motor imaginary with the aim of their transformation into commands for controlling computer systems. For this purpose, one can use the methods based on the registration of event-related potentials [22], methods of machine learning and artificial intelligence [23, 24], techniques for isolating the time–frequency structure of the signals [25] and methods for restoring connections between different brain areas using multichannel data [19, 26]. At the same time, these techniques demonstrate positive results for trained subjects. Wolpaw and McFarland showed that their participants were able to move a cursor with two-dimensional control after several sessions of training [27]. In [28], the effects of motor imagery on \(\mu /\alpha \)- (8–12 Hz) and \(\beta \)- (18–25 Hz) rhythm activity were studied, compared to those of actual movement.

For untrained subjects, the BCI is the more challenging task, however, less studied [29]. Existing motor imagery classification algorithms being applied to untrained participants do not always achieve good performances because of the noisy and nonstationary nature of the EEG signals and inter-subject variability [30]. We expect that this problem may be solved by using approaches of nonlinear dynamics, which were successfully applied in neurodynamics, including neuronal signal processing [31]. In particular, such techniques were used to measure entropy of biological signals [32], reveal features of neural synchronization [33], quantify power-law statistics of EEG signals [34], estimate EEG complexity based on fractal dimension [35], etc.

In this paper, we analyze spatio-temporal and time–frequency characteristics of electrical brain activity associated with real and imaginary motor actions in the group of untrained subjects, using different methods of nonlinear dynamics. At the first stage, we apply multifractal formalism to the analysis of EEG data to reveal brain areas demonstrating the most significant differences between real motor actions and imaginary movement. Then, using time–frequency analysis we study in detail the structure of the considered brain areas with the aim to reveal characteristic oscillatory patterns, which occur in different brain areas and interact with each other when the motor action (or imagination) takes place. Finally, we propose an automatic algorithm, which allows online detection of the observed patterns, and experimentally verify its efficiency.

2 Materials and methods

2.1 Subjects and recordings

Twelve healthy volunteers including both males and females, between the ages of 20 and 43, participated in the experiments. All of them signed a written consent. The experimental studies were performed in accordance with the Declaration of Helsinki and approved by the local research Ethics Committee of the Yurij Gagarin State Technical University of Saratov.

The multichannel EEG was recorded at 250 Hz sampling rate from \(P=19\) electrodes with two reference electrodes placed at the standard positions of the 10–20 international system [36]. To register EEG data, we used cup adhesive Ag/AgCl electrodes placed on “TIEN–20” paste. Immediately before the experiment start, all the necessary procedures to increase the conductivity of the skin and reduce its resistance were carried out with the use of abrasive “NuPrep” gel. The impedances were monitored when the electrodes were installed and measured during experiments. Usually, the impedance values varied within \(2\div 5\) k\(\Omega \). The ground electrode N was located above the forehead and two reference electrodes \(A_{1,2}\) were located on mastoids. The EEG signals were filtered by a band-pass filter with cutoff points at 1 Hz (HP) and 100 Hz (LP) and a 50-Hz notch filter. The electroencephalograph “Encephalan–EEGR–19/26” (Taganrog, Russian Federation) with multiple EEG channels was used for amplification and analog-to-digital conversion of the EEG signals. In addition to 10–20 electrode system, the signals from frontal electrode Fpz and occipital electrode Oz were used. Electroencephalograph “Encephalan–EEG–19/26” possesses the registration certificate of the Russian Federation Federal Service for Supervision in Health Care No. FCP 2007/00124 of 07.11.2014 and the European Certificate CE 538571 of the British Standards Institute (BSI).

Fig. 1
figure 1

Experimental design. a Two types of tasks: (left) real movement of the right arm \((\mathrm {RAM})\) and (right) imaginary movement of the right arm \((\mathrm {IAM})\). b The structure of experimental sessions: \(\mathrm {RAM}_i\) and \(\mathrm {IAM}_i\) (\(i=\overline{1,M}\) being the number of individual event in the session) define a single real and imaginary movement, respectively. \(M=20\) is the total number of events in the session, \(\Delta t=4\) s is the time interval reserved for the task. Each session is preceded by a video message with instructions and each event in the session is preceded by an audio message. \(\mathrm {RE}_j\) and \(\mathrm {IM}_j\) (\(j=\overline{1,N}\) being the session number) correspond to the sessions in which the real and imaginary movements take place, respectively; \(N=5\) is the total number of sessions, associated with each type of movement. The experiment starts with a 5-min background EEG recording (\(\mathrm {BCG}_1\)) and ends with a 5-min background recording (\(\mathrm {BCG}_2\))

Full size image

2.2 Experimental procedure

The experimental procedure is schematically illustrated in Fig. 1. Each of twelve participants was subjected to one experiment, lasting approximately 30 min. During the experiment a volunteer was instructed to perform two types of tasks: to lift slowly the right hand (in the shoulder joint) \((\mathrm {RAM})\) and imagine such a movement during a given time interval \((\mathrm {IAM})\) (Fig. 1a). The whole experiment was split into 10 sessions, 5 sessions (\(\mathrm {RE}_j\)) of real movements and 5 imaginary movements (\(\mathrm {IM}_j\)). Each \(\mathrm {RE}\) session followed by a \(\mathrm {IM}\) session. The experiment started with a 5-min background EEG recording (\(\mathrm {BCG}_1\)) and ended with a 5-min background recording (\(\mathrm {BCG}_2\)) (Fig. 1b). Each session was preceded by a short visual message with instructions and contained \(M=20\) identical events. Each event in the session was preceded by a short sound message and should be performed within a reserved time interval \(\Delta t=4\) s (Fig. 1c). The experiments were carried out during the first half of the day at a specially equipped laboratory where the volunteer was siting comfortably and effects of external stimuli, e.g., external noise and bright light, were minimized.

2.3 Signal analysis

Wavelet transform modulus maxima (WTMM) The WTMM method proposed by Muzy et al. [37] was applied to estimate the complexity of the EEG signals. This method is now widely used to describe complex scaling phenomena in nonstationary time series [38]. It gives the singularity spectrum of a signal x(t) based on the continuous wavelet transform

$$\begin{aligned} W(a,\tau )=\frac{1}{a} \int _{-\infty }^{\infty }x(t)\psi \left( \frac{t-\tau }{a}\right) \mathrm{d}t \end{aligned}$$
(1)

with parameters a and \(\tau \) characterizing the scale and translation of wavelet function \(\psi \). In the multifractal analysis, real-valued wavelets are mainly used, such as the MHAT function

$$\begin{aligned} \psi (t)=(1-t^2) \exp \left( -\frac{t^2}{2}\right) . \end{aligned}$$
(2)

Near the singularity point \(t^*\), the power-law dependence of the wavelet coefficients

$$\begin{aligned} W(a,t^*) \sim a^{h(t^*)} \end{aligned}$$
(3)

takes place with the Hölder exponent h. The value of h characterizes the strength of the singularity and varies along the signal x(t). In order to provide a statistical analysis of singularities, the approach based on the partition functions is mainly applied. It assumes the extraction of all skeleton lines (the lines of local maxima of |W(a, t)|) and the construction of the functions

$$\begin{aligned} Z(q,a) = \sum _{l \in L(a)} \left( \sup _{a' \le a} |W(a', t_l(a'))| \right) ^q \sim a^{\tau (q)} \end{aligned}$$
(4)

with L(a) being a full set of skeleton lines at the scale a, \(t_l(a)\) being the position of the maximum related to the line l and \(\tau (q)\) being the scaling exponents.

The values \(\tau (q)\) can be found by the analysis of the dependence (4) in the double-logarithmic plot. After their estimation, the Hölder exponents h(q) and the singularity spectrum D(h) are obtained as

$$\begin{aligned} h(q)= & {} \frac{{rm d}\tau (q)}{{rm d}q}, \end{aligned}$$
(5)
$$\begin{aligned} D(h)= & {} qh-\tau (q). \end{aligned}$$
(6)

The function D(h) is the Hausdorff dimension D of singularity points characterized by the exponent \(h(t)=h\). The position of the singularity spectrum is determined by the mean Hölder exponent \(H=h(0)\), while the width of the singularity spectrum quantifies the degree of inhomogeneity of the analyzed data. It is often used as a complexity measure of nonstationary processes.

WTMM combines complexity analysis and correlation analysis representing a powerful tool for studying nonstationary and inhomogeneous processes. Application of wavelet transform at the first stage of the method allows ignoring polynomial trends presented in the analyzed data. As a result, there is no need to preprocess a experimental EEG data before applying the WTMM method. When computing the singularity spectrum D(h), two main measures are considered: the width of this spectrum characterizing the degree of inhomogeneity (complexity) of data and the mean Hölder exponent H reflecting correlation properties. The performed analysis has revealed main distinctions in the second quantity. It should be noted that WTMM provides a faster convergence of estimated quantities as compared with the standard correlation analysis and, therefore, a better characterization of complex processes based on short data sets. The latter enables an improved quality of separation between different physiological states for limited amounts of physiological data [39, 40].

Time–frequency analysis is based on the continuous wavelet transformation (1) where complex-valued Morlet wavelet was chosen as the mother function

$$\begin{aligned} \psi (\eta )=\pi ^{-1/4}\mathrm{e}^{i\omega _0\eta }\mathrm{e}^{-\eta ^2/2}, \end{aligned}$$
(7)

with \(\omega _0=2\pi \) being the central frequency of the Morlet and \(i=\sqrt{-1}\).

The wavelet energy spectrum \(E(t,f)=W^2(t,f)\) was calculated in the frequency band \(f\in [1,30]\) Hz (\(f=1/a\)). For each EEG channel, the values of whole wavelet energy \(E_{\mathrm {RE}}\), \(E_{\mathrm {IM}}\), \(E_{\mathrm {BCG}}\) associated with motor execution, motor imaginary and background EEG, respectively, were calculated by averaging E(t, f) indicated frequency band and over the experimental sessions (RE), (IM) and (BCG)

$$\begin{aligned} E_{\begin{array} {l} \mathrm {RE}\\ \mathrm {IM}\\ \mathrm {BCG} \end{array}}=\int \limits _{\begin{array} {l} t\in \mathrm {RE}\\ t\in \mathrm {IM}\\ t\in \mathrm {BCG} \end{array}}\left[ \int \limits _{1\mathrm {Hz}}^{30\mathrm {Hz}}E(t',f') \mathrm{d}f'\right] \mathrm{d}t'. \end{aligned}$$
(8)

In the frequency ranges of \(\delta \)-band (1–5 Hz), \(\mu /\alpha \)-band (8–13 Hz) and \(\beta \)-band (10–30 Hz), the energy values \(E_\delta (t)\), \(E_\mu (t)\) and \(E_\beta (t)\) were calculated for each EEG channel by averaging the value E(t, f) over the corresponding frequency band

$$\begin{aligned} E_{\mu ,\beta ,\delta }(t)=\int \limits _{\begin{array} {l} f\in \mu -\mathrm {band}\\ f\in \beta -\mathrm {band}\\ f\in \delta -\mathrm {band} \end{array}}E(t,f') \mathrm{d}f'. \end{aligned}$$
(9)

Empirical mode decomposition (EMD) [41] was used to extract the low-frequency trend from the temporal evolution of the spectral energy. This very recent method of time–frequency analysis of complex nonlinear and nonstationary signals is a promising tool for studying experimental EEG signals, already demonstrated its efficiency [42, 43]. One of the most important advantages of EMD is that the basic functions are not predetermined, like in Fourier or wavelet analysis; they are derived from the analyzed signal itself highly dependent on the structure of the initial signal. This feature makes the EMD very convenient and highly adaptive for signal analysis.

The EMD method allows the decomposition of the initial signal (for example, EEG) into a sum of amplitude-modulated components with a zero mean value referred to as empirical modes (EM). Algorithm of EMD for some signal x(t) includes the following steps:

  1. 1.

    Finding all extrema (minima and maxima) on signal x(t)

  2. 2.

    Interpolation of signal between minima and maxima and construction of two envelopes: \(e_\mathrm{min}(t)\) and \(e_\mathrm{max}(t)\)

  3. 3.

    Calculation of low-frequency component of signal (trend) m(t):

    $$\begin{aligned} {m(t)=\frac{e_\mathrm{min}(t)+e_\mathrm{max}(t)}{2}} \end{aligned}$$
    (10)
  4. 4.

    Extraction of high-frequency component of signal (empirical mode) d(t):

    $$\begin{aligned} {d(t)=x(t) - m(t)} \end{aligned}$$
    (11)
  5. 5.

    Reiteration of steps 1–4 for trend m(t) for calculation of subsequent empirical mode.

Steps 1–4 of the EMD algorithm allow to calculate the first EM. In order to calculate the second, third, and subsequent EMs, all steps must be repeated for, respectively, first, second, etc., trends m(t) [instead of the initial signal x(t)] [43].

Example of EMD for EEG signal x(t) is illustrated on Fig. 2. Figure 2 contains short fragment of the initial EEG signal x(t) and first four EMs, calculated for this fragment. Each of the EMs is characterized by its own frequency range, total number of EMs and their frequency ranges are highly dependent on the initial signal. The first EM has the highest frequency, and the latter becomes lower with the growing number of the EM. Frequency ranges of different EMs mostly correspond to different oscillatory patterns on the signal. In order to extract the desired low-frequency trend from the signal x(t), one must consider the EM with lowest frequency.

Fig. 2
figure 2

Example of empirical mode decomposition for typical EEG signal x(t) registered by the Cz channel

Full size image

3 Results of off-line signal analysis

3.1 Multifractal analysis

The experimental results were analyzed using the multifractal analysis. Mean Hölder exponents were calculated for each 4-s EEG trial, associated with motor execution \(H_\mathrm {RE}\) and motor imaginary \(H_\mathrm {IM}\). The obtained values were then averaged over 100 EEG trials, and mean values \(\overline{H}_\mathrm {RE}\) and \(\overline{H}_\mathrm {IM}\) were used to characterize real and imaginary movements. The background activity was analyzed by averaging the mean Hölder exponent \(\overline{H}_\mathrm {BCG}\) over the 5-min background EEG (\(\mathrm {BCG}_1\)), preceding the motor-related experimental sessions (Fig. 1 b).

The performed estimations of the singularity spectra revealed significant distinctions between real and imaginary hand movements reflected in the position of D(h), i.e., in the mean Hölder exponent H illustrated in Fig. 3.

Fig. 3
figure 3

a Mean Hölder exponents related to real (RE) and imaginary (IM) movements of the right arm, estimated from EEG channel “Cz” for all subjects. b Hölder exponents related to real (RE) and imaginary (IM) movements and background EEG (BCG) calculated from EEG channels Cz and C3 and averaged over all subjects. All data are shown as mean ± SE

Full size image

The graphs in panel (a) show the values of the mean Hölder exponents related to real (RE) and imaginary (IM) movements of the right arm, estimated from the EEG channel Cz for all subjects and averaged over a large number (\(N=100\)) of events. The error bars indicate the standard error, which quantifies the deviation of the mean Hölder exponent within individual events. One can see that despite essential intra-group variability, an increase in \(\overline{H}\) for real movements was observed for all subjects. Similar results were obtained while considering other EEG channels. Thus, the performed multifractal analysis allowed us to distinguish between real and imaginary arm movements independently of the selected EEG channel.

Along with the distinctions between motor execution and motor imaginary, the possibility to extract the brain state associated with both ME and MI from the background EEG is also of great interest. For this purpose, we considered differences in the mean Hölder exponents corresponding to motor execution \(\overline{H}_\mathrm {RE}\), motor imaginary \(\overline{H}_\mathrm {IM}\) and background EEG \(\overline{H}_\mathrm {BCG}\). We found that the difference between motor execution and background EEG activity was well pronounced in all channels, while the distinction between background EEG and motor imaginary depended on the electrode position. As an example, in Fig. 3b we plot the values of the mean Hölder exponents related to real (RE) and imaginary (IM) movements and background EEG (BCG) calculated from EEG channels Cz and C3 (the data are averaged over all subjects and the bars define the standard error). One can see that for the Cz channel, the mean Hölder exponent increased for motor execution and decreased for motor imaginary. The statistical analysis based on paired t test showed that both these changes were judged as significant \((p<0.05)\). At the same time, the mean Hölder exponent calculated from the C3 channel demonstrated significant \((p<0.05)\) changes for motor execution, but changes in H-values between background EEG and motor imaginary were found to be insignificant \((p=0.12)\). Thus, according to Fig. 3b imaginary movements of all subjects can be effectively classified from background EEG based on the Hölder exponents calculated from the Cz channel, while the difference between imaginary movements and background EEG extracted the C3 channel was insignificant.

In Fig. 3c, we illustrate the differences between the mean Hölder exponents corresponding to real/imaginary movement and background EEG calculated from each EEG channel, distributed over the brain surface in accordance with electrode positions and averaged over all subjects. The results provide the evidence that the most significant differences between ME and background EEG and between MI and background EEG in untrained subjects took place in the frontal brain area. Motor execution resulted in a significant increase in the mean Hölder exponents calculated from EEG channels located in the frontal brain area, whereas motor imaginary led to a decrease in \(\overline{H}\)-values.

The obtained results revealed a significant influence of the frontal brain areas on the mean Hölder exponents calculated for the EEG segment associated with motor execution and motor imaginary. At the same time, the analysis of the EEG data from other brain areas, e.g., temporal lobe and central lobe, did not reveal any changes in the Hölder exponents. This allowed us to conclude that these areas did not play a significant role in generation of motor-related brain rhythms. This is a surprising result, because many previous papers reported the involvement of these brain structures in motor activity [44, 45].

The observed behavior of the Hölder exponents can be explained by the complex interaction between different oscillatory patterns in these brain areas. In order to clarify the nature of such interactions, we performed the time–frequency analysis.

3.2 Time–frequency analysis

The time–frequency analysis was based on the continuous wavelet decomposition. In this case, unlike the estimation of a singularity spectrum, the complex-valued Morlet wavelet (7) was used instead of the real MHAT function (2). The wavelet energy spectrum W(f, t) was calculated within the frequency band \(f\in [1,30]\) Hz. In the first stage, the values of energy associated with real (\(E^i_\mathrm {RE}\)), imaginary (\(E^i_\mathrm {IM}\)) movements and background EEG (\(E^i_\mathrm {BCG}\)) were calculated for each EEG trial (each 4 s length) and averaged over 100 trials. The obtained values were then averaged over the whole frequency band \(f\in [1,30]\) Hz. As the result, the set of values (\(E_\mathrm {RE}\), \(E_\mathrm {IM}\), \(E_\mathrm {BCG}\)) characterizing the energy of the electrical brain activity of each subject during real and imaginary movements and background EEG was obtained from every EEG channel (8).

In the first stage, the differences between motor execution/imaginary and background EEG were judged based on the calculation of the coefficients \(\langle E_\mathrm {RE}{-}E_\mathrm {BCG}\rangle \), \(\langle E_\mathrm {IM}{-}E_\mathrm {BCG} \rangle \), \(\langle E_\mathrm {RE}{-}E_\mathrm {IM} \rangle \) describing the change of the whole spectral energy associated with the considered states. These values were calculated for each EEG channel and averaged over the group of participants.

Fig. 4
figure 4

a Differences between the values of wavelet energy calculated for real movement \(E_\mathrm {RE}\) and background EEG \(E_\mathrm {BCG}\). b Imaginary movement \(E_\mathrm {IM}\) and background EEG \(E_\mathrm {BCG}\). c Real \(E_\mathrm {RE}\) and imaginary \(E_\mathrm {IM}\) movements. The values of energy (E) were calculated by averaging W(f, t) over the frequency band from 1 to 30 Hz, time interval \(t=4\) s and 100 trials. \(\langle \dots \rangle \) denotes group averaging over all subjects

Full size image

Figure 4 displays the revealed differences distributed over the brain surface. One can see that compared to background EEG, real movements were characterized by an increase in the energy over the most of the EEG channels (Fig. 4a). Imaginary movements, on the contrary, resulted in a decrease in the energy in the corresponding brain areas (Fig. 4b). In particular, real movements were associated with a significant increase in the energy in frontal and temporal lobes, whereas imaginary movements with a decrease in the energy in the same lobes. As the result, real and imaginary movements in untrained subjects can be effectively distinguished by analyzing frontal and temporal EEG signals. These results of energy analysis correlated with the behavior of the Hölder exponents (Fig. 3b).

As we already mentioned above, it is surprising that according to the Hölder exponents and the whole spectral energy, the neural structures of parietal and central lobes were not involved in real and imaginary movements. In order to deeply understand the role of parietal and central lobes in the considered types of activity, the detailed time–frequency structure of EEG signal was analyzed.

Fig. 5
figure 5

Time–frequency plots of changes in wavelet energy E(f, t), \(f\in (1,30)\) Hz, \(t\in (0,4)\) s associated with a real and b imaginary movements with respect to the background EEG. Presented data were averaged over 100 trials and shown for each of 21 EEG channels. Red and blue colors indicate the time–frequency plane for which the energy value respectively increased and decreased during real or imaginary movements. Color saturation shows the degree of changes. (Color figure online)

Full size image

In Fig. 5, we present the time–frequency dependencies which illustrate the changes in the value of wavelet energy E(f, t), \(f\in (1,30)\) Hz, \(t\in (0,4)\) s associated with (a) real and (b) imaginary movements with respect to the background EEG. Presented data were averaged over 100 EEG trials recorded from all 21 EEG channels of one subject. The time–frequency plots are labeled and located on the head-like layout, according to the position of the recording electrodes. Red and blue colors indicate time–frequency plane for which the energy value respectively increased and decreased during real or imaginary movement. Color saturation shows the degree of changes.

One can see that a significant increase for real movement and a significant decrease for imaginary movements of wavelet energy in frontal and temporal lobes were mainly determined by the low-frequency (1–5 Hz) \(\delta \)-waves. The energy of \(\delta \)-waves exhibited a significant change in frontal area, which decreased rapidly while moving from frontal to parietal lobe. This resulted in a significant change in wavelet energy. At the same time, the time–frequency structure of EEG signal was much more complicated. The features of real and imaginary movements were characterized by the transition and distribution of the energy between different frequency bands. Considering real movement (Fig. 5a), one can see that electrical activity in temporal lobe together with an increase in \(\delta \)-waves’ amplitude was characterized by a decrease in the wavelet energy for \(f\in [8,12]\) Hz (\(\mu \)-rhythm) and decreasing energy in \(f\in [15,30]\) Hz (\(\beta \)-activity). This effect is known in the scientific literature as event-related desynchronization (ERD) [46]. The ERD associated with motor activity was previously observed in the frequency bands of 8–10, 10–12, 12–20 and 20–30 Hz [47]. It is known that motor execution is characterized by both event-related desynchronization and event-related synchronization (ERS). The ERD was usually observed in \(\alpha \) (or \(\mu \)) and \(\beta \)-bands [48], while such effect in \(\delta \)-band was much less studied [49]. At the same time, according to Fig. 5a, event-related synchronization of \(\delta \)-activity took place during motor execution together with event-related desynchronization of \(\mu /\alpha \))- and \(\beta \)-rhythms.

In Fig. 5a, the colored areas A and B indicate the brain segments where the considered event-related behavior was the most pronounced. One can see that the observed ERD in \(\mu \)-rhythm prevailed in temporal, central and parietal lobes (colored area A), where the motor area took place [50]. It should be noted that this area is shifted from the center to the left side, which is connected with hemispheric asymmetry, associated with arm movements [51]. An accompanying event-related increase in the amplitude of the low-frequency \(\delta \)-activity was more pronounced in frontal lobe (shaded area B).

Motor imaginary, in turn, was also associated with significant changes in \(\delta \)-activity, which can be observed in frontal area (colored area B in Fig. 5b). However, in this case the energy of \(\delta \)-waves decreased, that was associated with event-related desynchronization. While ERD took place in \(\delta \)-band, \(\mu \)-rhythm exhibited event-related synchronization, which was well pronounced in the most areas of the brain, but reached maximal value in central and parietal lobes and significantly decreased in temporal lobes (colored area A in Fig. 5b).

As we already mentioned above, the brain activity associated with the motor functions was characterized by the interaction between different brain rhythms. This process is illustrated in detail in Fig. 6 for the case of motor execution. In Fig. 6a, one can see typical EEG traces recorded in parietal and occipital lobes during a single experimental session (RE) associated with alternating motor executions. In Fig. 6b, the location of the spectral components characterized by the maximal energy is shown for the considered EEG recordings. In Fig. 6c, the mean values of the spectral energy averaged over \(\alpha \)-band \(\langle {E}\rangle _{\alpha }\), \(\delta \)-band \(\langle E\rangle _{\delta }\) and the set of EEG signals are shown. It is seen that the motor execution was accompanied by the transition of the spectral energy between \(\alpha \)- and \(\delta \)-bands. It was demonstrated by both the location of the most pronounced spectral components and changes in mean (band-related) spectral energy. In Fig. 6b, one can see that the spectral components with the high energy appeared in \(\delta \)-band during motor execution and disappeared in \(\alpha \)-band, and vice versa; such components occurred again in \(\alpha \)-band after the motor execution finished. At the same time, the values of spectral energy \(<E>_{\alpha }\), \(<E>_{\delta }\) calculated for these bands evolved in anti-phase, i.e., the value of \(\langle E\rangle _{\alpha }\) decreased during motor execution and \(\langle E\rangle _{\delta }\) increased (Fig. 6c).

It is important to note that, according to Fig. 6, the characteristic features associated with motor executions in \(\alpha \)- and \(\delta \)-bands can be identified for each single event. This was exciting since the most studies on motor execution and imaginary judged ERS and ERD from the data obtained by averaging over a large number of events. In this context, our results show that for a single event, changes in the motor-related energy in \(\alpha \)- and \(\delta \)-bands can be extracted from EEG trials by averaging them over particular brain areas. One can expect that taken into account the identified brain areas, where the effects of ERD and ERS are the most pronounced (Fig. 5), it is possible to detect a single motor execution or imagination in real time.

Fig. 6
figure 6

a Set of EEG signals recorded from occipital and parietal lobes during experimental session RE when alternating real movements were performed. The vertical dashed lines indicate the moments, when the actions start. The considered brain area is shown by the shadow. b Location of most pronounced spectral components and changes in mean (band-related) spectral energy

Full size image

4 Online detection of real and imaginary movements

In previous section, we demonstrated that \(\delta \)-band (\(f\in [1,5]\) Hz) in combination with \(\mu /\alpha \)-band (\(f\in [8,13]\) Hz) can be effectively used for the extraction of the features of the brain activity associated with motor execution and motor imaginary in untrained subjects. Having considered the time–frequency structure of EEG signals averaged over 100 trials associated with the motor actions/imaginary, we found that during the motor execution the event-related desynchronization in \(\mu /\alpha \)-band took place in temporal, central and parietal lobes, and event-related synchronization in \(\delta \)-band was most pronounced in frontal lobe. During motor imaginary, \(\mu /\alpha \)-band exhibited event-related synchronization which was mainly revealed in central and parietal lobes and significantly decreased in temporal lobes. Similar to motor execution, motor imaginary was characterized by a significant change in frontal lobe \(\delta \)-activity, while ME was associated with ERS in \(\delta \)-band, and MI was characterized by ERD in \(\delta \)-band.

Based on the obtained results, in this section we propose a real-time algorithm, which is able to extract a single event associated with motor execution or motor imaginary, from the background EEG. The proposed algorithm is based on the consideration of the wavelet energy in \(\delta \)- and \(\mu /\alpha \)-bands and includes the following steps:

  1. 1.

    Selection of two brain areas for which the ERD and ERS are the most pronounced.

  2. 2.

    Calculation of the wavelet energy in \(\delta \)- and \(\mu /\alpha \)-bands and averaging it over EEG channels belonging to selected areas.

  3. 3.

    Consideration of the evolution of these energy values in time and extraction of the low (\(<0.2\) Hz) frequency component via the empirical mode decomposition.

  4. 4.

    Numerical differentiation of the obtained dependencies in time and extracting the cases when these dependencies exhibit anti-phase increase and decrease.

  5. 5.

    Extraction of motor-related activity with the help of predefined thresholds.

Fig. 7
figure 7

Flowchart of online MI/ME detection algorithm. Flows (a)–(c) are defined the different options of the algorithm evaluation: (a) corresponds to background EEG without ME or MI events detection, (b) either MI or ME is detected, (c) algorithm skips EEG data due to predefined idle time interval caused by motor execution or motor imaginary acts

Full size image

Figure 7 illustrates the flowchart of proposed online MI-ME detection algorithm. One can see that algorithm can process three different situations: (1) neither ME nor MI is detected since the corresponded EEG features are not found (flow (a) in Fig. 7); (2) either MI or ME is detected when the considered control characteristics satisfy corresponding conditions (flow (b) in Fig. 7); (3) algorithm is not detected for ME and MI events due to passing EEG traces immediately followed by the already detected onset of MI or ME during predefined idle time (flow (c) in Fig. 7). In this case, the algorithm skips EEG data points during the idle time \(t_\mathrm{idle}=1\) s.

Fig. 8
figure 8

Extraction of features of EEG signals associated with a single motor execution and b motor imagination. Upper panel: (right) colored areas A and B used for estimation of changes in \(\mu \)- and \(\delta \)-activity, respectively, and (left) typical EEG trials recorded in these areas. Lower panel: (upper traces) \(E^\mathbf A _\alpha \) and \(E^\mathbf B _\delta \) corresponding to wavelet energy, averaged over \(\mu \) (or \(\alpha \))- and \(\delta \)-bands, respectively, calculated from the set of EEG trials recorded from areas A and B. \(\mathrm {EM}_4^\mathbf A \) and \(\mathrm {EM}_4^\mathbf B \) are the empirical modes of the \(4\mathrm {th}\) kind, calculated for the dependences \(E^\mathbf A _\alpha (t)\) and \(E^\mathbf B _\delta (t)\), respectively. (\(\mathrm {EM}_4^\mathbf A \))\(^{\prime }\) and (\(\mathrm {EM}_4^\mathbf B \))\(^{\prime }\) define the time derivative of the empirical modes \(\mathrm {EM}_4^\mathbf A \) and \(\mathrm {EM}_4^\mathbf B \). The shaded areas highlight the cases when \((\mathrm {EM}_4^\mathbf A )^{\prime }<0\) and \((\mathrm {EM}_4^\mathbf B )^{\prime }>0\) (for real movements) and \((\mathrm {EM}_4^\mathbf A )^{\prime } >0\) and \((\mathrm {EM}_4^\mathbf B )^{\prime }<0\) (for imaginary movements). The pulses (detection markers) indicate the moments of time when the motor execution or motor imagination took place. (Color figure online)

Full size image

Figure 8 demonstrates how the proposed algorithm shown in Fig. 7 was applied in a single session for which (a) the alternating motor executions and (b) motor imaginations were performed. In the right-hand side of the upper panels in Fig. 8a, b, we color the areas A and B from which the EEG signals were recorded and used for estimation of changes in \(\mu \)- and \(\delta \)-activity, respectively. These areas were used according to the results of the time–frequency analysis described in Sect. 3.2. Thus, for motor execution \(\mu /\alpha \)-activity was analyzed in central, parietal and temporal lobes, while for motor imaginary in frontal, central, parietal and occipital lobes. \(\delta \)-activity was analyzed in frontal lobe in both cases. In the left-hand side of the upper panels of Fig. 8a, b, typical EEG traces recorded from these brain areas are shown. The vertical dashed lines indicate the time moments at which the motor execution/imagination started. The values of \(E^\mathbf A _\alpha \) and \(E^\mathbf B _\delta \) correspond to the wavelet energy, averaged over \(f\in \mu (\alpha )\) and \(f\in \delta \) bands, respectively, and over the set of EEG traces belonging to areas A and B. \(\mathrm {EM}_4^\mathbf A \) and \(\mathrm {EM}_4^\mathbf B \) are the empirical modes of the \(4\mathrm {th}\) kind calculated for the dependencies \(E^\mathbf A _\alpha (t)\) and \(E^\mathbf B _\delta (t)\), respectively. (\(\mathrm {EM}_4^\mathbf A \))\(^{\prime }\) and (\(\mathrm {EM}_4^\mathbf B \))\(^{\prime }\) define the time derivative of the empirical modes \(\mathrm {EM}_4^\mathbf A \) and \(\mathrm {EM}_4^\mathbf B \). The shaded areas highlight the cases when \((\mathrm {EM}_4^\mathbf A )^{\prime }<0\) and \((\mathrm {EM}_4^\mathbf B )^{\prime }>0\) (for real movements) and \((\mathrm {EM}_4^\mathbf A )^{\prime } >0\) and \((\mathrm {EM}_4^\mathbf B )^{\prime }<0\) (for imaginary movements). These conditions correspond to an increase in \(\delta \)-activity and a decrease in \(\mu /\alpha \))-activity for real movement, and a decrease in \(\delta \)-activity and an increase in \(\mu /\alpha \)-activity for imaginary movement. The pulses in the lower panel of Fig. 8 indicate the moments of time when (a) motor execution or (b) motor imagination took place.

One can see that the proposed algorithm was able to identify the motor execution and motor imaginary from the background EEG. Having applied to the experimental session included 20 events of ME and 20 events of MI, our algorithm was able to correctly recognize 19 ME events and 16 MI events. The number of false events was 0 and 2, respectively, for these sessions.

Group analysis performed for 12 subjects demonstrated \(92.9\,\pm \,7.8\%\) ME events detected with \(5.5\,\pm \,4.2\%\) false alarm rate and \(81.6\,\pm \,6.2\%\) MI events detected with \(9.1\,\pm \,6.2\%\) false alarm rate. Motor executions were recognized in average not later than \(0.5\pm 0.2\) s after they had actually been started. Imaginary movements were sometime recognized \(1.9\pm 1.1\) s after they started, that likely was caused by the lack of training.

5 Conclusion

The event-related synchronization and event-related desynchronization are known to be associated with motor execution and motor imaginary. These effects were actively studied in the frequency bands of 8–10, 10–12, 12–20 and 20–30 Hz. At the same time, event-related desynchronization (ERD) and event-related synchronization (ERS) in low-frequency \(\delta \)-band remains poorly investigated in the case of motor-related brain activity, but known to be associated with decision-making [49].

In this paper, based on multifractal and time–frequency analysis we have shown that the effect of ERD and ERS in \(\delta \)-band along with \(\mu /\alpha \)) band was associated with motor execution and motor imagination.

The performed multifractal analysis has revealed significant distinctions between real and imaginary arm movements which were reflected in the value of the mean Hölder exponent. Considering the set of the Hölder exponents calculated from all EEG channels, we have found that the most significant distinctions between motor execution, motor imaginary and background EEG can be extracted in frontal lobe.

Having analyzed the time–frequency of the EEG signals, we have demonstrated that such behavior of the Hölder exponent was connected with the effect of ERD and ERS in \(\delta \)-band. We have shown that \(\delta \)-band (\(f\in [1,5]\) Hz) in combination with \(\mu /\alpha \)-band (\(f\in [8,13]\) Hz) can be effectively used for the extraction of the features of the brain activity, associated with motor execution and motor imaginary. We have found that during the motor execution, the event-related desynchronization in \(\mu /\alpha \)-band occurs in temporal, central and parietal lobes, and event-related synchronization in \(\delta \)-band was most pronounced in frontal lobe. During motor imaginary, \(\mu /\alpha \)-band exhibited ERS which was mostly seen in central and parietal lobes and significantly decreased in temporal lobes. It was accompanied by ERD in \(\delta \)-band, which was highly pronounced in frontal area.

We suppose that such EEG signatures of MI are caused by the lack of subject training. Taking into account [48], one can expect that in trained subjects the time–frequency and spatio-temporal structures of EEG signals, corresponded to imaginary movement, are very similar to once, corresponded to real movement. In this case, application of the proposed method is expected to reveal effects of ERD in \(\mu /\alpha \)- and \(\beta \)-bands instead of ERS, obtained in untrained subjects. In additional, it is expected that, unlike the real movement, ERS will take place in \(\delta \)-band, which is associated with cognitive activity, associated with decision-making [49], since subject has to decide every time either to move with the hand or not.

Based on the obtained results, we have proposed an algorithm enable to extract a single event associated with either motor execution or motor imaginary, from the background EEG.

We believe that the obtained results are of interest for fundamental science. They clarify the relationship between a singularity spectrum and a wavelet energy spectrum of EEG signals. In addition, the revealed motor-related features of EEG signals are valuable for neuroscience and other areas of science and technology aimed to understand brain properties and design brain–machine interface systems (BMI) [52, 53]. In this context, one knows that the efficiency of BMI is defined by the ability of the operator to generate certain stable EEG patterns. That means that the BMI is affected by operator proficiency and the inter-subject variability [29]. In this respect, our results suggest possibility to develop unified ME-MI classifier. This possibility in its turn can be applied for building brain–machine interface for multiple and untrained users [30].