Thought Chart: Tracking Dynamic EEG Brain Connectivity with Unsupervised Manifold Learning

Abstract

Assuming that the topological space containing all possible brain states forms a very high-dimensional manifold, this paper proposes an unsupervised manifold learning framework to reconstruct and visualize this manifold using EEG brain connectivity data acquired from a group of healthy volunteers.

Once this manifold is constructed, the temporal sequence of an individual’s EEG activities can then be represented as a trajectory or thought chart in this space. Our framework first applied graph dissimilarity space embedding to the temporal EEG connectomes of 20 healthy volunteers, both at rest and during an emotion regulation task (ERT), followed by local neighborhood reconstruction then nonlinear dimensionality reduction (NDR) in order to reconstruct and embed the learned manifold in a lower-dimensional Euclidean space. We showed that resting and ERT thought charts represent distinct trajectories, and that the manifold resembles dynamical systems on the torus. Additionally, new trajectories can be inserted on-line via out-of-sample embedding, thus providing a novel data-driven framework for classifying brain states, with potential applications in neurofeedback via real-time thought chart visualization.

1 Introduction

Inspired by the Nash embedding theorems [1, 2], which showed that any compact Riemannian n-manifold can be C¹ isometrically embedded in a Euclidean space of dimension 2n + 1, and by Theorema Egregium, which showed that the Gauss curvature of a 2-manifold embedded in 3D depends only on the first fundamental form and is thus invariant when it is bent without stretched or torn (i.e., complete isometric mappings preserve the Gauss curvature), the goal of this study is to understand the manifold properties or the intrinsic geometry of the mind’s topological space. We develop this framework around the conjecture that, at least with non-invasive functional brain imaging, this manifold is smooth and differentiable (i.e., local neighborhoods are homeomorphic to a Euclidean space with the same number of dimensions). Our conjecture ultimately relies on the intuition that at least on a macroscopic spatiotemporal scale brain dynamics are continuous, or simply put do not abruptly “jump” from one state to the next. To test this hypothesis, we utilized resting-state and task EEG data from healthy participants performing an emotional regulation task. We hypothesized that the reconstructed manifold will reflect different properties of the brain’s state at rest and during the performance of the task. Additionally, by sampling the space, we can extract specific aspects of the manifold that reflect task performance.

2 Methods

2.1 Subject Recruitment and Data Acquisition

EEG data were collected from 20 psychiatrically healthy participants (age: 27.2 ± 9.3) using the Biosemi system (Biosemi, Amsterdam, Netherlands) with an elastic cap with 34 scalp channels. Each participant underwent one recording session of an eight minute eye-open resting state and one separate session of Emotion Regulation Task (ERT). During ERT, participants were requested to look at pictures displayed on the screen, and listen to a corresponding auditory guide. Two types of pictures will be on display for seven seconds in random orders: emotionally neutral pictures (landscape, everyday objects, etc.) and negative pictures (car crash, nature disasters, etc.). The auditory guide will come after the picture on display for one second, instructing the participant to “look”: viewing the neutral pictures; to “maintain”: viewing the negative pictures as they normally would; or to “reappraise”: viewing the negative pictures while attempting to reduce their emotion response by reinterpreting the meaning of pictures [3, 4]. EEG data were preprocessed using Brain Vision Analyzer (Brain Products, Gilching Germany), by first segmenting task trials into 7 s segments with a window size of 0.05 s (the first and last 5 time points were discarded, resulting in 130 time points per task; resting state data was similarly preprocessed). Frequencies-of-interest were set from 1 Hz to 50 Hz in increments of 1 Hz. The final output of each subject was averaged over trials within the same task (Fig. 1).

Fig. 1.

An illustration of a typical ERT session. A fixation point is on display before each trial, then followed by either a neutral or negative picture on the screen. An audio instruction will ask test subjects to maintain, reappraise or stay neutral.

2.2 Weighted Phase Lag Index Based EEG Connectome

As functional communications between two brain regions result in synchronized or phase-coupled EEG readouts, in this study we used weighted phase lag index (WPLI) computed [5] between the times series of two channels to form EEG connectomes (each of which a symmetric 34 by 34 matrix). Mathematically, WPLI is defined as:

$$ WPLI_{xy} = \frac{{n^{ - 1} \mathop \sum \nolimits_{t = 1}^{n} |imag\left( {S_{xyt} } \right)|sgn(imag\left( {S_{xyt} } \right))}}{{n^{ - 1} \mathop \sum \nolimits_{t = 1}^{n} |imag\left( {S_{xyt} } \right)|}} $$

(1)

Where imag(Sxyt) indicate the cross-spectral density at time t in the complex plane xy, and sgn is the sign function (−1, +1 or 0) [5]. The connectivity matrices were generated with the MATLAB toolbox Fieldtrip (Donders Centre for Cognitive Neuroimaging, Nijmegen, Netherlands). The final output time-dependent EEG connectome for an individual task of each subject is arranged as 34 * 34 * 50 * 130 (channel * channel * frequency * time). Given several lines of evidence suggesting the role of theta EEG (4–7 Hz) in emotion regulation [6, 7] and our recent graph analyses further demonstrating distinct theta wave changes during ERT, in this study we primarily focused on the manifold informed by theta wave EEG connectomes.

2.3 Learning the Manifold with Graph Dissimilarity Space Embedding and Nonlinear Dimensionality Reduction (NDR)

In order to learn the intrinsic geometry of a high-dimensional manifold, one needs a sufficiently large amount of data points. Thus, we treat the EEG connectomes from all subjects at all time points as sampling possible states of the manifold that is shared among all subjects. Then, graph dissimilarity space embedding is used to represent each connectome as a point in a very high-dimensional space (number of dimensions equal to the number of prototype graphs as described below). This is then followed by (1) manifold learning via local neighborhood reconstruction and (2) manifold embedding into a lower dimensional Euclidean space using nonlinear dimensionality reduction (NDR). Once this is achieved, thought chart of any given individual can be constructed by tracing the trajectory of the time-dependent connectome of that subject for any given task.

Next, we describe the graph dissimilarity space embedding procedure [8, 9]. Let G = {G₁, …, G_n} be n “prototype” graph observations ${{\rm{G}}_{\rm{i}}} \in {\mathbb{G}}$ (the set of all possible graphs under consideration) and d a distance metric that can be computed between two graphs ${\rm{d}}:{\mathbb{G}} \times {\mathbb{G}} \to \left[ {0,\infty } \right)$, then any graph ${\rm{X}} \in {\mathbb{G}}$ can be represented using $\varphi_n^G:{\mathbb{G}} \to {{\mathbb{R}}^{\rm{n}}}$, defined as the n-dimensional vector $\varphi_{n}^{G} \left( {\text{X}} \right) = [{\text{d}}\left( {{\text{X}},{\text{ G}}_{ 1} } \right), \ldots {\text{d}}({\text{X}},{\text{ G}}_{\text{n}} )]$. Note here the number of dimensions is in the same order as the number of observations in the dataset (in this study all connectomes were used as prototypes).

Once connectomes are represented in this fashion, the next step of manifold learning is local neighborhood reconstruction. Here we emphasize that this step is crucial in order to properly learn the manifold’s intrinsic geometry, as d (which is used to define coordinates in the embedding space, and thus not intrinsic to the manifold) will not properly inform geodesics (the shortest paths on the manifold, which is an intrinsic property) except in local neighborhoods. While such a construction calls for a “good” choice of the distance function d, we posit that given a sufficiently large amount of data points the learned manifold will converge to the true manifold with any reasonably chosen d. Given two connectome matrices X and Y a natural choice, which we adopted here, is the Euclidean distance: $d\left( {X,Y} \right) = \sqrt {\sum\nolimits_{ij} {(X_{ij} - Y_{ij} )^{2} } }$ and ${ \varphi }_{\text{n}}^{\text{G}} ({\text{X}}) - { \varphi }_{\text{n}}^{\text{G}} ({\text{Y}}) = \sqrt {\sum\nolimits_{\text{i}} {({\text{d}}({\text{X}}, {\text{G}}_{\text{i}} ) - {\text{d}}({\text{Y}}, {\text{G}}_{\text{i}} ))^{2} } }$.

Once local neighborhood is learned, the next step is to reduce the manifold that is currently in a very-high dimensional space (recall this dimension is the number of prototype graphs used, here in the order of 10⁴) and further embed it in a more manageable lower-dimensional space. Using the prototypical isometric embedding procedure isomap as an example, this step thus entails the computation of geodesics based on neighborhood information followed by (quasi-) isometric embedding of the geodesics.

Here, let us pause for a moment and point out the resemblance between dissimilarity space embedding and Frechet’s classical isometric embedding argument, showing that any n-point (x ₁ , …, x _n ) metric space can be isometrically embedded in $l_{\infty }^{n - 1}$ [10, 11] by simply placing any point $x \in \left\{ {x_{ 1} , \ldots ,x_{n} } \right\}$ at the coordinates: d(x, x₁), d(x, x₂), … d(x, x_n−1) where d is the metric (interestingly, this result was later improved to $l_{\infty }^{n - 2}$).

2.4 Out-of-Sample Embedding

Once this manifold is constructed, a series of dynamic connectomes acquired from a new subject can then be embedded on-line if we exploit out-of-sample extensions for NDR techniques [12]. Again using isomap as an example (in this case the procedure is called landmark isomap [13]) where pairwise geodesics need to be approximated using neighborhood information followed by eigendecomposition of the resulting squared distance matrix, this is particularly relevant as this step turns out to be the bottleneck of the algorithm. Using out-of-sample embedding will thus allow us to precompute and store the dimensionally-reduced manifold representation and the corresponding embedding, with which we can then perform online computation given new observations.

In brief, in the case of isomap the second step relies on applying the classic multidimensional scaling (MDS) to the centered squared geodesic distance matrix $\Delta _{c} = \frac{ - 1}{2}H_{n}\Delta _{n} H_{n} ,$ whose eigendecomposition provides the basis for lower dimensional embedding. Mathematically, it can be shown that the n column vectors denoting coordinates for the n landmark points in a lower k-dimensional space is simply given by truncating the following matrix at the k-th row:

$$ L = \left( {\begin{array}{*{20}c} {\sqrt {\lambda_{1} } v_{1}^{T} } \\ \vdots \\ {\sqrt {\lambda_{n} } v_{n}^{T} } \\ \end{array} } \right) $$

(2)

Here the eigenvalues $\lambda_{i}$ are arranged from high to low, while $\Delta _{n}$ is the squared geodesic matrix of the landmark points and the centering matrix $H_{n} = I - \frac{1}{n}11^{T}$.

Then the out-of-sample embedding of any new observation can be obtained by first forming the column vector $\delta = (\delta_{ 1} ,\delta_{ 2} , \ldots \delta_{n} )^{T}$ that stores this new point’s squared geodesic distances to all pre-embedded observations in the training dataset, followed by forming the “interpolated” embedded coordinates: $\frac{ - 1}{2}L_{k}^{\# } \left( {\delta - \overline{{\delta_{n} }} } \right)$.

Here $\overline{{\delta_{n} }}$ is the mean of the n column vectors in $\Delta _{n}$ and $L_{k}^{\# }$ the pseudoinverse of truncated at the k-th row:

$$ L_{k}^{\# } = \left( {\begin{array}{*{20}c} {v_{1}^{T} /\sqrt {\lambda_{1} } } \\ \vdots \\ {v_{k}^{T} /\sqrt {\lambda_{k} } } \\ \end{array} } \right) $$

(3)

3 Results

After averaging across theta frequencies (4–7 Hz) and combining both resting and ERT theta connectomes for all time points, 20 healthy subjects thus contributed a total of 10400 connectomes (130 * 20 * 4). Using the classic isomap (local neighborhood of each connectome operationally defined as its 30 nearest neighbors; the number of dimensions reduced from 10400 to 3), the reconstructed theta-EEG manifold exhibited a principal dimension that is shared by all 4 states (x-axis in Fig. 2; also see a front view of the manifold in Fig. 4) with a secondary small-amplitude rotation around it. Visually, this manifold thus resembles the shape of a snake by spiraling around its main axis. Moreover, the amplitude of the rotation follows an ordered transition: (from low- to high- amplitude) resting (red), neutral (green), maintain (purple) and reappraise (blue), corresponding to increasing cognitive load of the tasks. Insets of Fig. 2 further show the corresponding embedding using locally linear embedding (LLE [14]), which exhibits a similar rotation-along-main-axis shape (LLE is another prototype NDR technique that is however non-isometric), and the embedding generated using simple PCA (a linear technique) that does not recover the complex shape seen in either isomap or LLE.

Fig. 2.

An example thought chart during reappraise learned from the temporal EEG connectomes of 20 healthy subjects, both at rest and during ERT, using NDR methods of Isomap and LLE, as well as standard PCA. Visually, NDR methods yielded a rotation around the manifold’s principal dimension (x-axis), with the amplitude of rotation following an ordered transition from resting, neutral, maintain to reappraise.

Using out-of-sample embedding, the mean group though chart for neutral, maintain and reappraise (computed by averaging, for each time point, theta EEG connectomes across all subjects for that task) can also be embedded and visualized (note while mathematically doable, it is inappropriate to compute mean resting thought chart as resting state is not stimuli-evoked). Interestingly, there exists a similar ordered antero-posterior transition from neutral, maintain to reappraise, indicating that the posterior section of the manifold (more negative along the x-axis) represents states that require higher cognitive demands (Fig. 3).

Fig. 3.

Out-of-sample embedding of the mean group thought chart for neutral, maintain, and reappraise (note that we cannot time-average the resting-state thought chart across subjects). Similar to Fig. 2 there is an ordered antero-posterior transition from neutral, maintain, to reappraise.

To further understand theta-EEG connectome dynamics, we additionally studied the four distinct sub-regions of the manifold (i.e. segments of the “snake”): the head (primarily resting), the mid body (primarily neutral), the posterior body (a mixture of neutral, maintain and reappraise) and the tail (primarily maintain and reappraise; Fig. 4). Sampling these segments reveals marked connectome differences. Analysis of the top 10 edge strengths in the head region (Fig. 4a) demonstrated increased theta coupling in fronto-parieto-occipital leads while the body (neutral-predominant, Fig. 4b; maintain/reappraise dominant, Fig. 4c) is characterized with predominantly increased theta coupling between occipital leads. Last, the tail (maintain/reappraise only, Fig. 4d) revealed increased theta coupling between frontal and parietal leads. Thus, the manifold comprises subspaces representing resting, visual processing (common feature of neutral, maintain and reappraise) and cognitive control (distinct feature of maintain and reappraise). Edge strength analyses of the manifold-sampled EEG connectomes demonstrated increased patterns of theta coupling that are highly consistent with previous reports of frequency-band coupling associated with the resting-state [15], visual processing [16], and cognitive control [17].

Fig. 4.

Mean 34 * 34 theta EEG connectomes of four distinct segments of the Neurospace: the head (a), the mid and posterior body (b, c) and the tail (d) (left). For each mean connectome, its ten strongest edges were visualized on the layout of the electrodes (right).

4 Discussion and Conclusion

In this study we proposed a novel unsupervised manifold learning framework to construct a state space, in the form of a manifold embedded in 3D that quasi-isometrically visualizes EEG connectome dynamics. Moreover, in this space one can visualize time-dependent brain activities as a trajectory or thought chart. We applied this approach to a group of healthy controls, both at rest and during tasks, and showed that the reconstructed manifold exhibits a complex and highly structured geometry, with distinct sub-regions corresponding to different mental states. Our results suggested that the manifold has a principal dimension that is primarily linear, and a rotation around this principal dimension whose amplitude increases with cognitive demands.

In this context, this manifold resembles dynamical systems on the torus [18] (the surface of a doughnut), in that trajectories are generated by the product of two circles: the large torus circle corresponding to the principal dimension while the small or minor circle corresponding to the secondary rotation around it (and that cognitive demands change the ratio between the radii of the two circles).

Limitations of our approach merit further discussion. First, as a quasi-isometric technique isomap aims to preserve the pairwise geodesics on the manifold, i.e., approximating global isometry when the embedding is constrained to a given dimension. By contrast other classes of local NDR methods such as LLE unfold the manifold by preserving local linear reconstruction relationship (i.e., local parameterization) of each point within its neighborhood. Moreover, as the Theorema Egregium only guaranteed the invariance of Gauss curvature for complete isometric embeddings of 2-manifolds, it is unclear if the manifold constructed using one NDR technique is necessarily more “correct”. Nevertheless, both LLE and isomap recover a principal dimension and a rotation around it, while simple linear techniques such as PCA did not. We thus posit that the highly structured complex geometry recovered using our framework may indeed inform the hidden properties of brain dynamics and the underlying neurophysiological mechanisms that generate them.