Introduction

Magnetoencephalography (MEG) is a noninvasive technique to investigate neural activity with a high temporal resolution (Hamalainen et al. 1993). Source estimation methods, such as minimum-norm estimation (MNE), can be applied to map the sensor space measures to sources in the brain. Recently, methodological advances have made real-time MEG source estimation possible (Sudre et al. 2011). This approach can be used to better understand brain function, to identify mental states (Jones et al. 2010; Ziegler et al. 2010), and to create more effective brain-computer interface (BCI) systems (Soekadar et al. 2011; Besserve et al. 2011). Monitoring the activity at the source level enables, for example, the adaption of the experiment (feedback), real-time adjustment of neuro implants, and real-time monitoring of drug effects Michel et al. (1995).

Two major challenges in real-time source localization are the low signal-to-noise ratio (SNR) of single-trial as well as moving-average measurements and the limited time available for computations. Chowdhury et al. (2013) recently published a data-driven method for parceling the cortical surface based on a pre-source localization, which determines seed points to create parcels for each measurement setup. This set of functionally homogeneous active areas results in a smaller gain matrix and as a result the inverse problem becomes better conditioned.

In this work, we present a novel model-driven clustering method for real-time source localization to handle the low SNR and to reduce the computational cost. Our method is based on two assumptions. First, it is assumed that the neural activity is spatially smooth and can be organized into cortical parcels (Chowdhury et al. 2013; Dinh et al. 2012). This spatial smoothness justifies that a whole region can be activated in the same quantity as a region’s representative dipole. Second, it is assumed that a low SNR reduces the number of distinguishable sources (Supek and Aine 1993; Tarkiainen et al. 2003). These assumptions lead to different possibilities to obtain a reduced inverse operator. Based on the comparison of these approaches, we propose to cluster the dense gain matrix (Dinh et al. 2012) based on a cortical atlas (Destrieux et al. 2010) into a reduced matrix which maps neural activity to parcellated cortical regions.

To realize complex real-time processing chains requires a highly generic and scalable software framework. For this purpose we utilize our in-house real-time acquisition and processing solution MNE-X, which is designed for the acquisition and processing of real-time data streams from multiple sensors. MNE-X is part of our EEG/MEG data processing software MNE-CPP which we made recently available (Dinh et al. 2013) under an open source license. The software package can be downloaded at: https://github.com/mne-tools/mne-cpp Footnote 1

Methods

Gain Matrix Reduction

The relationship of MEG signals and their sources is

$$\begin{aligned} y = G q + \epsilon , \end{aligned}$$
(1)

where the vector \(y\) contains the signals measured at the sensors, \(G\) is the gain matrix, \(q\) contains the amplitudes of dipole sources, and \(\epsilon\) is the noise. The column vector triplet of \(G\) mapping the dipole triplet at the \(r\) th (cortical) site to the sensors will be denoted by \(g_r\). In our approach, the dipoles were approximately evenly distributed on the white matter surface.

Many source estimates can be expressed as the solution of the minimization problem

$$\begin{aligned} \hat{q} = \arg \min _{q} \left( \left\| y - Gq \right\| + f(q) \right) , \end{aligned}$$
(2)

where the first term expresses a (weighted) norm of the differences between the measured signals and those predicted by the model while \(f(q)\) incorporates the a priori assumptions. One popular solution is the minimum-norm estimate (MNE) (Hamalainen and Ilmoniemi 1994). In this approach, f(q) is the squared \(L_2\)-norm of the currents, weighted by a regularization parameter \(\lambda ^2\), and the solution is obtained by multiplying the data with the linear inverse operator:

$$\begin{aligned} {\hat{q}} = My, \end{aligned}$$
(3)

where

$$\begin{aligned} M = G^T\left( GG^T+\lambda ^2 C\right) ^{-1}. \end{aligned}$$
(4)

In the above, C is the noise covariance matrix and \(\lambda ^2\) is the regularization parameter adjusting the relative weight of the two terms in Eq. 2.

One way to set the regularization parameter is to relate it to the SNR of the whitened data. This approach is described in detail by Lin et al. (2006).

MNE is fast to compute, it is thus potentially useful for real-time source localization. The main goal of this work was to improve the capability of MNE to handle low SNRs and to further accelerate the computation for real-time source localization.

To achieve these goals, we represent regional cortical activity by a small number of dipoles. We thus preserved dipoles which could still be distinguished in the presence of low SNRs. At the same time, we reduced the computational complexity. To justify the reduction, it had to be assumed that neural activity is spatially smooth and can be organized into parcels, representing anatomical units, which in many cases have a functional meaning. It was further assumed that nearby dipoles produce very similar MEG signal patterns.

We considered four methods to reduce the gain matrix. The most obvious method was a selection of a smaller set of spatially evenly distributed dipoles, which reduces the computational effort and improves the condition of the inverse problem at the same time. We compared this selection method to three different clustering methods, which are based on clustering either a high-resolution gain matrix \(G\) or directly the inverse operator \(M\). To our knowledge, the clustering approach has not been used in real-time source localization before.

Clustering Methods

By reducing the gain matrix through clustering, we aimed at the region-wise calculation of the most representative dipoles instead of the arbitrary selection of dipoles which are not ensured to be representative. In order to preserve the spatial and anatomical relations when clustering the gain matrix, an anatomical brain atlas was used to divide the brain into several anatomical units, which often have a functional meaning, see Fig. 1.

Fig. 1
figure 1

The Destrieux’s brain atlas (Destrieux et al. 2010), which divides the cortex into 74 anatomical, often related to functional, units in each hemisphere

Full size image

We employed three different clustering methods: (I) a clustering based on the gain matrix \(G\), (II) a combined clustering based on \(G\) and the inverse operator \(M\), and (III) a clustering directly based on the inverse operator \(M\).

Method I

According to the atlas, the gain matrix \(G\) can be divided into a set of \(s\) sub-gain matrices:

$$\begin{aligned} G = \{G_1, \ldots , G_s\}. \end{aligned}$$
(5)

Dipoles belonging to these region-related subsets \(G_i\) were clustered into \(k_i\) clusters using the k-means algorithm (Lloyd 1982).

Prior to clustering, we whitened the gain matrix with the spatial noise covariance matrix \(C\) estimated from the baselines of evoked response data:

$$\begin{aligned} \widetilde{G} = C^{-1/2} G. \end{aligned}$$
(6)

The whitening takes into account different units of measure (planar gradiometers vs. magnetometers in the Elekta Neuromag® VectorView™) and different noise levels in the channels. After whitening, the gain vectors were clustered region by region:

$$\begin{aligned} K(\widetilde{G_i}) = \sum ^{n_{\widetilde{G_i}}/k_i}_{r=1}{\min _{j\in {1 \ldots k_i}} \left\| \widetilde{g_r} - \mu _j \right\| _p }\ \ (p = 1,2) . \end{aligned}$$
(7)

The aim of k-means \(K(\widetilde{G_i})\) was to calculate the centroid \(\mu _j\), which was assumed to be a representative dipole of a cluster, which consists of \(n_{G_i}\) dipoles belonging to one or a part of one region. The \(L_1\)-norm \(\left( p = 1\right)\) and the \(L_2\)-norm \(\left( p = 2\right)\) were used and compared as two separate distance measures within the k-means algorithm. All k-means cluster indices forming separate dipole centroids can be concatenated to one cluster operator \(D_{\widetilde{G}}\), which clusters the original gain matrix \(G\) at once:

$$\begin{aligned} G^*= G D_{\widetilde{G}} \end{aligned}$$
(8)

and the corresponding inverse operator is:

$$\begin{aligned} M_D = D^T_{\widetilde{G}} G^T (G D_{\widetilde{G}} D^T_{\widetilde{G}} G^T + \lambda ^2 C)^{-1}. \end{aligned}$$
(9)

In the following, we refer to this method as \({\text{I}}_{\text{L1}}\) and \({\text{I}}_{\text{L2}}\), depending on the norm employed in clustering, see Table 1.

The number of clusters \(k_i\) for each region depends on the number of dipoles \(n_{G_i}\) in each region. It is determined by:

$$\begin{aligned} k_i = \min \left\{ k \in \mathbb {Z}| k_i = \lceil \frac{n_{G_i}}{z} \rceil \right\} , \end{aligned}$$
(10)

where the cluster size \(z\) represents a reduction constant. It specifies the number of dipoles clustered to one centroid. The optimal constant \(z\) is calculated with the help of the condition number \(\kappa\) of the clustered gain matrix \(G^*(z)\) (Sect.  3.1):

$$\begin{aligned} \kappa (G^*(z)) = \left| \frac{\sigma _{\max }(G^*(z))}{\sigma _{\min }(G^*(z))} \right| , \end{aligned}$$
(11)

where \(\sigma _{\max }\) and \(\sigma _{\min }\) refer to the maximal or minimal singular value of the clustered gain matrix \(G^*\), respectively. A lower condition number \(\kappa\) represents better conditioned data. Different cluster sizes \(z\) and k-means distance measures yield different condition numbers \(\kappa\).

The optimal reduction constant \(z\) corresponds to the minimal condition number:

$$\begin{aligned} z_{opt} = \arg \min _{z\in \mathbb {Z}}(\kappa (G^*(z))). \end{aligned}$$
(12)

Method II

In this approach, we applied the cluster indices \(D_{\widetilde{G}}\) (Eq. 6) of Method I to \(M\), thus effectively clustering the source estimate values based on the full gain matrix:

$$\begin{aligned} D^T_{\widetilde{G}} M = D^T_{\widetilde{G}} G^T (G G^T + \lambda ^2 C)^{-1}. \end{aligned}$$
(13)

This left the inner term of Eq. 9 untouched:

$$\begin{aligned} G D_{\widetilde{G}} D^T_{\widetilde{G}} G^T. \end{aligned}$$
(14)

In analogy with method I, this method is referred to as \({\text{II}}_{\text{L1}}\) and \({\text{II}}_{\text{L2}}\) depending on whether the \(L_1\)- or \(L_2\)-norm is used in clustering, see Table 1.

Method III

In this method the clustering was based on the source waveforms, i.e., the unclustered inverse operator \(M\), instead of the gain matrix \(G\) and we thus obtained the inverse operator

$$\begin{aligned} D^T_{M^T} M = D^T_{M^T} G^T (G G^T + \lambda ^2 C)^{-1}. \end{aligned}$$
(15)

In consequence the source space was taken as ground truth, no matter how sensitive the sensors were to a particular dipole location.

Method IV

In method IV (the selection method), the reduced gain matrix \(G^*\) was created by selecting a subset of equally distributed dipoles of the original gain matrix \(G\). The selected dipoles then acted as representations of the anatomical units each of them belongs to (Fig. 1). This helped to interpret the result even though the source space was highly decimated. Furthermore the result was comparable to the clustering method described in the previous paragraph. In the following, we refer to the selection method by IV, see Table 1.

Table 1 Overview of different inverse operators evaluated
Full size table

Comparison

The different methods were evaluated by comparing point spread and crosstalk computed from the resolution matrix

$$\begin{aligned} R = M G. \end{aligned}$$
(16)

The point spread is a measure on how focal the source estimation is, i.e., how strong neighbored sources are activated together with the actual active source. On the other hand, the crosstalk is a measure for the activation across the whole source space including non-neighbored sources.

The columns of \(R\) describe the point spread across regions when a given source in the dense source grid is active. The relative average point spread \(\psi\) was calculated by combining all values belonging to one region, i.e., source activities of one region, divided by the activities of all regions:

$$\begin{aligned} \psi = 1 - \frac{1}{{\text{card}}(\forall j \in \nu _k)} \frac{\sum \limits _{o\in \nu _k} \sum \limits _{j\in \nu _k} R_{o j} }{ \sum \limits _{j\in \nu _k} \left( \sum \limits _{o\in \nu _k} R_{o j} + \sum \limits _{o\notin \nu _k} R_{o j} \right) }. \end{aligned}$$
(17)

The current region is referred to as \(\nu _k\). A lower value \(\psi\) refers to a decreased point spread and is therefore better.

The relative crosstalk \(\chi\), caused by the activity of one region towards all other vertices can be found among the rows of \(R\). Within one row, the vertices belonging to one cluster \(\nu _k\) are put into relation to all vertices.

$$\begin{aligned}&\chi = \frac{\alpha }{\alpha +\beta } \nonumber \\&\alpha = \frac{1}{{\text{card}}(\ {\forall j \notin \nu _k})} \sum \limits _{j \notin \nu _k} R_{kj}^2\nonumber \\&\beta = \frac{1}{{\text{card}}(\forall j \in \nu _k)} \sum \limits _{j \in \nu _k} R_{kj}^2 \end{aligned}$$
(18)

A lower relative crosstalk has to be preferred.

The Real-Time Processing Chain

The real-time processing chain was realized with two applications of the MNE-CPP software package Dinh et al. (2013), see Sect. 1 and Fig. 2.

Fig. 2
figure 2

Acquisition chain: Data were collected with mne_rt_server directly from the medical or emulated medical device. Multiple clients, e.g., MNE-X, can be connected to the real-time server

Full size image

The first application is mne_rt_server, with a plug-in based architecture to realize several different medical device connectors. This application converts the measured data to a standardized real-time fiff data stream. Fiff is the file format used by Elekta Neuromag® to store acquired data. The mne_rt_server fiff data stream can be accessed over a TCP/IP connection.

The second application is MNE-X, written in Qt C++ (Digia Plc 1991–2014), to acquire and process the data. It provides the functionality to control the MEG acquisition, to record the acquired data, and to process them in real-time. It has a generic plug-in architecture which can easily be extended. The real-time source localization is realized as an algorithm plug-in, see Fig. 3.

Fig. 3
figure 3

The source localization processing chain; (a) acquired data stream; independent averaging (b 1) or single-trial (b 2) and covariance C estimation (b 3); (c) Source localization which estimates sources \({\hat {q}}\) using the averaged data yave and the continuously updated inverse operator M.

Full size image

For real-time localization of evoked responses we employed a moving average of \(N_e\) responses. We assumed that the fraction of time occupied by the evoked response is small enough so that we could use the raw data directly to estimate the noise covariance matrix \(C\).

A new noise covariance matrix \(C\) was calculated every \(n_C\) samples and the inverse operator was updated accordingly.

We also computed a noise-normalized estimate (dynamic statistical parametric mapping—dSPM) (Dale et al. 2000) to reduce the location bias of the estimates and to decrease the dependency of the point-spread function on the source location.

Data Acquisition

As described in Sect. 2.2 mne_rt_server provides several plug-ins, including the Elekta Neuromag® MEG VectorView™ system plug-in and the MEG system emulator plug-in. Since there is no difference in the processing pipeline between having an actual subject inside the MEG room or using a previously recorded raw file as a subject emulation, we used the emulation plug-in to evaluate the developed algorithm and to find the optimal adjustments. The emulation configuration, mne_rt_server together with the emulation plug-in run on the Neuromag acquisition workstation, allows the emulation of the study as it would be carried out in real-time. The emulated data had a sampling frequency of \(1250\,{\text{Hz}}\).

Data of five subjects of the MIND multi-site MEG study (Weisend et al. 2007; Ou et al. 2007) using two different auditory recordings for each subject were used to evaluate the four reduction methods. In the auditory task of this study, three pure tones of different frequencies (500, 2000, and 4000 Hz) were presented to obtain a tonotopic map. The tones were randomly presented with an average inter-stimulus interval of \(1157\,\pm \,891\,{\text{ms}}\). The stimuli were delivered to the subject’s ear canal using sound transducers connected with plastic tubing to ergonomically designed earplugs. An attenuator was used to adjust the intensity of the tones. The hearing thresholds were determined individually for each subject before the session. The figures in the following sections were created using one data set of this study.

Subsequently, we conducted a measurement with the Elekta Neuromag® MEG system plug-in to confirm the real-time performance.

Wehner et al. (2008) described the influence of head movement of children in MEG on source localization. They used an Elekta Neuromag® MEG VectorView™ system as well. It turned out that source localization is relatively robust to head movement. It appeared that the frontal cortex is most prone to localization errors, with a worst case mean error of \(12\;{\text{mm}}\). Localizations within the auditory cortex are more robust towards head movement. Therefore, we did not include motion correction in the data processing.

The anatomical model was based on individual MRI data collected with a Siemens Magnetom® Avanto \(1.5\,{\text{T}}\) MR scanner. Four different sequences were acquired: two MPRAGE sequences with a receiver band width of 240 and 650, and two FLASH sequences with a flip angle of 5 and 30 \(^\circ\). The two MPRAGE sequences were used to reconstruct the motion corrected anatomical structures including the brain atlas which was performed with FreeSurfer (Fischl 2012). The volume conductor modeling was done with the MNE suite (Gramfort et al. 2013) using the FLASH sequences creating a three layer (outer skin; outer skull; inner skull) boundary element model. The gain matrix was calculated for approximately 7500 uniformly distributed dipoles on the white matter surface.

Evaluation

We determined the best performing selection / clustering method by comparing point spread and crosstalk (Eqs. 17, 18). These two measures were analyzed using nonparametric statistics, since the underlying distribution was not known. Friedman’s (1937) test was used to determine whether the distribution underlying the crosstalk and point spread results are significantly different. In a next step the Wilcoxon–Mann–Whitney test (Wilcoxon 1945; Mann and Whitney 1947) was applied to unveil significant differences between each single method. To counteract the multiple comparison problem we employed the Bonferroni correction, which reduces the test threshold \(p_t\) given by the significance level p by the number of pairwise comparisons \(n_c\):

$$\begin{aligned} p_t = \frac{p}{n_c}. \end{aligned}$$
(19)

We took the condition as a measure for the robustness of the inverse problem towards noisy data, i.e., the inverse problem using a better conditioned gain matrix (Eq. 11) is less influenced by low SNRs (Eichardt et al. 2012). The optimal cluster size was found by evaluating the condition of different sized gain matrices, see Eq. 11. Moreover, the condition was taken as a figure of merit to reveal whether the \(L_1\)- or \(L_2\)-norm performs better in clustering. The influence of clustering on the distribution of the dipole sensitivity was analyzed subsequently. The cluster centers of \(G^*\) were mapped to the location of the most similar dipole of the original gain matrix \(G\) to obtain their coordinates. We also evaluated the source localization precision of the method with the lowest point spread and crosstalk. A dipole fit based on the MIND data (see Sect. 2.3) was calculated as reference localization. The dipole fit was applied to the baseline (\(-100\;{\text{to}}\;0\;{\text{ms}}\) pre-stimulus interval) corrected, 1–40 Hz band-pass filtered, averaged data, which contained all available trials of the left auditory evoked 500 Hz stimulus fields. The dipole fitting was performed in the Elekta Neuromag® Xfit software.

The localized ipsi- and contralateral dipoles were mapped to their closest corresponding Destrieux’s brain region, see Fig. 1, which were taken as reference regions. In the next step, we calculated MNE employing the reduced gain matrix \(G^*\) and \(N_e = 1,\, 2,\, 4,\, 10,\, {\text{and}}\, 20\). Each localization was repeated 31 times using different random selections of trials. The smallest distance between the reference dipole and the most active localized region was taken as localization error, i.e., the root-mean-square error was calculated between the reference dipole and the vertex of the respective region which was closest to the reference dipole. The most active region was determined based on the source activity of a single time point (\(0.12\;{\text{s}}\)) or the average of the source samples between 0.116–0.124, 0.113–0.127 ms or 0.11–0.13 ms. The average of source samples was calculated to smooth the estimated source activity, which was highly transient caused by noise. The error measure was determined for each hemisphere.

Finally, we verified the real-time performance using the Elekta Neuromag® MEG VectorView™ system. The overall fixed delay \(t_{{\varDelta }}\), caused by the processing chain was calculated based on the buffer sizes \(n_{\text{buf}}\) and the given sampling rate \(f_s\):

$$\begin{aligned} t_{{\varDelta }} = \frac{n_{\text{buf}}}{2} / f_s. \end{aligned}$$
(20)

Results

Gain Matrix Reduction

Crosstalk and point spread (Eq. 16) were computed to determine the best selection / cluster method among those listed in Table 1. The results for a fixed number of approximately \(416\) centroids (\(z=20\), see cluster size investigation, next paragraph) over all subject recordings are shown in Figs. 4 and 7, representing histograms of the respective parameter in vertical direction. Friedman’s test on the crosstalk medians (Fig. 5) of each recording showed with \(p=1.3495\times 10^{-6}\) a significant difference among the methods given a significance level of \(p=0.05\). The method-wise comparison of the crosstalk medians using the Wilcoxon-Mann-Whitney test and the Bonferroni correction showed that the medians of the selection method (IV) and the original dense gain matrix were significantly different to the other methods’ medians (Table 2). The selection method (IV) and the original dense gain matrix had the highest crosstalk across all subjects whereas the cluster based reduction methods (I–III) had similar low crosstalk values (Fig. 4).

Fig. 4
figure 4

Relative crosstalk distribution over all subjects and measurements using different methods (Table 1) as well as the result of the original dense gain matrix. The violin plot shows a symmetric histogram of atlas regions. The length of the interval bars indicates the number of contained regions/vertices. Mean values are indicated by red bars (Color figure online)

Full size image
Fig. 5
figure 5

Distribution of methods’ crosstalk medians over all subjects and measurements

Full size image
Table 2 Method-wise Wilcoxon–Mann–Whitney test on the crosstalk given a \(p=0.05\) significance level
Full size table

In accordance with the crosstalk analysis we firstly analyzed the point spread (Fig. 6) with Friedman’s test which revealed a significant difference of \(p = 5.7442\times 10^{-9}\) between the method’s point spread median distributions given a significance level of \(p=0.05\). In the next step we also calculated the method-wise median comparison using the Wilcoxon-Mann-Whitney test in connection with a Bonferroni correction. The results showed that clustering the inverse operator (method \({\text{III}}_{{\text{L}}_1}\)) and the original dense gain matrix were significantly different across all subjects, see Table 3. The original dense gain matrix showed the highest point spread followed by method \({\text{III}}_{{\text{L}}_1}\) whereas the other methods performed similarly well (Fig. 7).

Fig. 6
figure 6

Distribution of method’s point spread medians over all subjects and measurements

Full size image
Table 3 Method-wise Wilcoxon-Mann-Whitney test on the point spread given a \(p=0.05\) significance level
Full size table
Fig. 7
figure 7

Relative point spread over all subjects and measurements of different selection / cluster paradigms (Table 1). The violin plot shows a symmetric histogram of dipoles of regions. The length of the interval bars indicates the number of contained dipoles. Mean values are indicated by red bars (Color figure online)

Full size image

Since the methods \({\text{I}}_{{\text{L}}_1}\) and \({\text{I}}_{{\text{L}}_2}\) had the lowest computational costs, see Table 1, an improvement in crosstalk of about \(10\%\) compared to method (IV), see Fig. 4, and the point spread of method \({\text{I}}_{{\text{L}}_1}\) was as good as others, we decided to choose method \({\text{I}}_{{\text{L}}_1}\) as the best performing method.

In the above we used a cluster size which resulted in a reduced gain matrix containing approximately 416 dipoles. In the following, we show how we determined the optimal number of clusters. The best cluster size \(z\) was determined using the condition number \(\kappa\) (Eq. 11) as a figure of merit. Figure 8 illustrates that the condition number has at the beginning a large descent with increasing cluster sizes. This changes at a certain point back into a small increase with larger cluster sizes. We empirically chose a cluster size \(z\) of 20, which seemed to be a good trade-off between a low condition number and a small clustered gain matrix for an accelerated computation. This value reduced our gain matrix \(G\) containing approximately 7500 dipoles to a gain matrix \(G^*\) containing approximately 416 dipoles.

Fig. 8
figure 8

Cluster size \(z\) and corresponding condition number \(\kappa\). The smallest, meaning the optimal, condition number can be found for a cluster size containing about 11 dipoles. The applied k-means clustering uses a \(L_1\)-norm distance measure

Full size image

Figure 9 visualizes the singular values of the full gain matrix \(G\) and the clustered gain matrices \(G^*\) using the \(L_1\)- and the \(L_2\)-norm. The k-means clustering with the \(L_1\)-norm resulted in an improved condition whereas it decreased with the \(L_2\)-norm. The \(L_1\)-norm clustered gain matrix \(G^*\) had fewer dipoles revealing a stronger separation from each other compared to the original gain matrix \(G\).

Fig. 9
figure 9

Singular values of the full gain matrix \(G\) and the clustered (\(L_1\) & \(L_2\)) gain matrices \(G^*\). The singular values were used to determine the condition of the gain matrices

Full size image

The mapping of the cluster centers of \(G^*\) to the closest dipole of the original gain matrix \(G\) allowed to obtain their coordinates. The distribution of these dipoles is illustrated in Fig. 10. It can be seen that the distribution almost uniformly covers the whole brain. The average values of the sensitivities in the original gain matrix \(G\) were \(-0.29\,\pm \,30.59 \,mu T/Am\) for the gradiometers and \(-0.04\,\pm \,44.67\, \mu T/Am^2\) for the magnetometers. The mapping of the centroids of \(G^*\) caused a deviation of \(-0.06\,\pm \,5.97\,\mu T/Am\) for the gradiometers and \(-0.05\pm 5.26\,\mu T/Am^2\) for the magnetometers. Since this difference is relatively high, the centroids were used to construct \(G^*\) instead of the closest dipoles of \(G\).

Fig. 10
figure 10

Cluster dipoles (large dots) mapped to the closest dipole of the original gain matrix (small dots; only shown for the precentral gyrus). Mapped centroids of the precentral gyrus are colored differently, referring to the original dipoles (small dots) forming one centroid. (Color codes do not correspond to the atlas of Fig. 1)

Full size image

Noise Covariance Estimation

The noise covariance matrix was estimated in real-time and was used to calculate a new inverse operator \(M\), which was therefore adapted to the present noise. The ergodicity was taken as description to determine the number of samples for a valid estimation of the noise covariance matrix.

The target ergodicity value change of less than \(5\%\) was reached when an interval of \(4\,{\text{s}}\) was used. Within our measurements we applied a sampling frequency of \(1250\,{\text{Hz}}\), which resulted in a new covariance matrix estimation every 5000 samples. The data segments did not overlap.

Source Localization

First, the reference localization data were created by averaging all available 99 left ear auditory stimulus responses of one exemplary subject, which were subsequently baseline corrected and band-pass filtered, as described in Sect. 2.4. The reference data is shown in a topographical layout in Fig. 11.

Fig. 11
figure 11

Topographic plot for a Elekta Neuromag® MEG VectorView™ sensor configuration of the reference auditory evoked field. Data were baseline corrected and band-pass filtered

Full size image

A dipole fit for each hemisphere was calculated as reference localization, see Sect. 2.4. The results are shown in Fig. 12.

Fig. 12
figure 12

The dipole fit, which was used as reference source localization. The dipole fit was created with the Elekta Neuromag® Xfit source localization toolbox. T - post stimulus time; r - location in the head coordinate system; gof - goodness-of-fit

Full size image

The reference dipoles were then mapped to the closest atlas region in both hemispheres. The obtained reference regions refer to the superior temporal sulcus for the ipsi and anterior transverse temporal gyrus (of Heschl) for the contra lateral hemisphere.

Method \({\text{I}}_{{\text{L}}_1}\), using MNE (see Table 1), was now computed in the clustered source space. The localization error between reference and localized region was calculated as described in Sect. 2.4. The results are shown in Fig. 13.

Fig. 13
figure 13

This Figure shows the localization error between reference and localized most active region. The most active region for each hemisphere was determined based on the source activity of one time point (0.12 s) or the average of source samples within an interval (0.116–0.124 s; 0.113–0.127 s; or 0.11–0.13 s).

Full size image

The results for a sample-wise reconstruction were lowest for localizations based on 10–20 averages. For the following analysis we decided to consider averages including 10  epochs, which allowed to have a good trade-off between the aim to monitor changes in the activity and the necessary noise reduction for a higher localization precision. One localization result based on the average of 10 randomly selected epochs from the example data set is shown in Fig. 14, which shows the waveforms of the sources over time as well as the mapped source activity at the N100 of the left auditory stimulus.

Fig. 14
figure 14

N100 dSPM localization result of a left auditory stimulus using 10 averages. The upper part visualizes the cluster activation of all clusters over time. The corresponding topography on the white matter surface for the time point 12 s after the stimulus is shown at the bottom

Full size image

Finally, we verified the real-time delays with the Elekta Neuromag® VectorView 306™ MEG device. We were able to verify that localization results of the MNE can be provided in real-time. Sampling rates up to 1250 Hz and a clustered gain matrix mapping 416 regions to 306 MEG sensors were handled by the processing chain. The internal sample buffer of the MEG device had a minimal size of at least 28 samples. That led to an average delay of \(11.2\,{\text{ms}}\) given a sampling rate of \(1250\,{\text{Hz}}\). The computation of MNE for all samples within a 10 ms window took only 3.36 \(\pm\) 0.97 ms and was, therefore, faster than the delay of the MEG system, which allowed to follow the data stream in real-time. This caused an overall average delay of approximately 15 ms, when the inverse calculation was directly applied to the real-time data stream.

Discussion

Real-time source localization involves two major challenges: low SNR and the limited time available for the computations. We addressed both issues by reducing the gain matrix. The computational cost was thus reduced and at the same time sources were easier to distinguish in the presence of noise, i.e., the inverse problem was better conditioned, and the crosstalk as well as the point spread were reduced.

We compared seven different reduction methods, see Table 1, and the original dense gain matrix. The results in Sect. 3.1 show that all gain matrix clustering methods led to a significant reduced crosstalk compared to selecting a fewer amount of dipoles or the original dense gain matrix. The point spread was reduced compared to method \({\text{III}}_{{\text{L}}_1}\) and the original dense gain matrix. In contrast to simple selection, the proposed clustering approach \({\text{I}}_{{\text{L}}_1}\) ensures that the reduced sampling of the source space is spatially and anatomically representative for the respective region.

Method \({\text{I}}_{{\text{L}}_1}\) reduces the gain matrix by calculating representative dipoles per parcellation, increasing the ability to distinguish regions even in the presence of a low SNR, which is indicated by the better condition (Fig. 8). This method had the lowest computational cost, showed a significant improved crosstalk compared to the selection method (IV) and the point spread performed similarly. Thus it was selected for the further examinations. The inverse calculation was applied to a reduced number of dipoles at the same time and could therefore be performed with ordinary workstations in real-time.

However, both real-time challenges were met at the cost of the resolution in space and sensitivity. First, only regions could be localized. Second, the k-means algorithm reduced the spread between the dipoles of the clustered gain matrix \(G^*\), increasing the condition \(\kappa\) of the clustered gain matrix \(G^*\) and therewith the ill-posedness of the inverse problem. This effect was even more prominent when the \(L_2\)-norm was applied, whereas the \(L_1\)-norm helped to preserve a stronger separation between the cluster centers.

The number of centroids per region was determined by the cluster size and was also evaluated. A large cluster size is favorable in terms of computation cost, but on the other hand possibly averages out dipole diversity. We determined the size by evaluating the condition for different cluster sizes (Fig. 8) from a full gain matrix with approximately 7500 dipoles. Also, taking into account the calculation speed, a cluster size containing 20 dipoles was favored.

The presented method showed a high localization precision when applied to raw data streams. Auditory responses on the cortex were localized with small error using averages of only four trials. The activation was correctly localized at and around the superior temporal gyrus. The comparison to a dipole reference localization was successful and showed good performance.

Since Wehner et al. (2008) found that source localization is relatively robust towards movement of the head, we did not apply a head movement correction in the real-time processing chain. However, an increase of the localization accuracy can be expected when head movement correction is applied. We are currently developing strategies of head movement corrections in the related BabyMEG project (Papadelis et al. 2013), which we will apply in future.

The results showed that the clustering performs well using the k-means algorithm. However, we are aware of the limits of k-means, i.e., dependency of the initialization, number of clusters must be known and the convex nature of the clusters. In future work, we will investigate further clustering algorithms, e.g., density-based spatial clustering of applications with noise (DBSCAN).

Babadi et al. (2014) recently proposed a clustering technique which uses the most significant eigenmodes of pre-computed Voronoi regions. In comparison to that, we used an anatomical brain atlas instead of Voronoi regions which allows to keep an anatomical, often related to a functional, representation of the cortex and provides an easily interpretable result for our real-time monitor. Furthermore, by calculating clusters using k-means we were able to cluster a free orientated gain matrix and were still able to calculate a reduced gain matrix. Another gain matrix clustering method was introduced by Chowdhury et al. (2013), which determines the clusters on a pre-localization basis and is therefore data driven. Here we propose a model-driven clustering which is based on the electromagnetic characteristics of the gain matrix. This allows us to conduct real-time localization without knowing the source estimates in advance.

Conclusion

We demonstrated that the selected real-time source localization approach can correctly locate sources of stimulus-locked activity in real-time. We showed that the source localization results can be calculated almost in real-time with an average delay of approximately 16 ms up to a sampling frequency of 1250 Hz for \(306\)-channel MEG data. We anticipate that the presented approach is suitable for several scenarios, e.g., the visualization and monitoring of ongoing activity to give a real-time feedback, to refine the localization within the most active regions followed by a subsequent high-resolution source localization, and to increase the accuracy of BCIs based on regional brain activity estimates.