Graph Edge Entropy in Maxwell-Boltzmann Statistics for Alzheimer’s Disease Analysis

Abstract

In this paper, we explore how to the decompose the global thermodynamic entropy of a network into components associated with its edges. Commencing from a statistical mechanical picture in which the normalised Laplacian matrix plays the role of Hamiltonian operator, thermodynamic entropy can be calculated from partition function associated with different energy level occupation distributions arising from Maxwell-Boltzmann statistics. Using the spectral decomposition of the Laplacian, we show how to project the edge-entropy components so that the detailed distribution of entropy across the edges of a network can be achieved. We apply the resulting method to fMRI activation networks to evaluate the qualitative and quantitative characterisations. The entropic measurement turns out to be an effective tool to identify the differences in the structure of Alzheimer’s disease by selecting the most salient anatomical brain regions.

1 Introduction

Functional magnetic resonance imaging (fMRI) provides a sophisticated means of studying the neuropathophysiology associated with Alzheimer’s disease (AD) [1]. It maps the network representation to neuronal activity between the various brain regions. The resulting network structure has proved useful in understanding Alzheimer’s disease (AD) via the analysis of intrinsic brain connectivity [2]. Although there are many tools to identify the affected brain regions in AD, it is still not clear how this abnormality affects the functional organization of the whole brain connection.

Tools from network analysis provide a convenient approach for understanding the functional association of different regions in the brain [2, 4]. The approach is to characterize the topological structures present in the brain and to quantify the functional interaction between brain regions, using the mathematical study of networks and graph theory. Graph theory offers an attractive route since it provides effective tools for characterizing network structures together with their intrinsic complexity [3, 6]. This approach has led to the design of several practical methods for characterizing the global and local structure of graphs [3]. Features based on the global and local measures of connectivity are widely used in functional brain analysis [5].

Unfortunately, there is relatively little literature aimed at studying structural network features using entropic analysis. The reason for that is the vast majority of techniques suggested by entropy provides a useful global characterisation of network structure, they do not lend themselves to the analysis of local measures of edge or subnetwork structure. However, entropy analysis is a more natural representation for brain structure, since they allow the information of activation signals for different anatomical structures in the brain.

This paper is motivated by the need to fill this important gap in the literature, and to establish effective methods for measuring the structural properties of entropy representing fMRI brain networks. In particular, in order to characterize the functional organization of the brain, our approach explores a novel edge entropy projection which can decompose the global network entropy computed from Maxwell-Boltzmann distribution [6, 12]. The new characterisations of edge entropy resulting from this analysis allow us to probe in finer detail the interactions between different anatomical regions in fMRI data from healthy controls and Alzheimer’s disease sufferers (AD).

The remainder of the paper is organized as follows. Section 2 briefly reviews the basic concepts in network representation, especially with a sophisticated study of von Neumann entropy. Section 3 reviews density matrix and Hamiltonian operator on graphs, and decompose the thermodynamic entropy on edges from Maxwell-Boltzmann statistics. Section 4 provides our experimental evaluation. Finally, Sect. 5 provides the conclusion and direction for future work.

2 Graph Representation

2.1 Preliminaries

Let G(V, E) be an undirected graph with node set V and edge set $E \subseteq V \times V$, and let V represent the total number of nodes on graph G(V, E). The adjacency matrix of a graph is A with the degree of node u is $d_u = \sum _{v \in V} A_{uv}$. Then, the Laplacian matrix is $L = D - A$, where D denotes the degree diagonal matrix whose elements are given by $D(u,u) = d_u$ and zeros elsewhere. The normalized Laplacian matrix $\tilde{L}$ of the graph G is defined as $\tilde{L} = D^{-\frac{1}{2}}LD^{\frac{1}{2}}$, and the spectral decomposition is $\tilde{L} = \varPhi \tilde{\varLambda } \varPhi ^T$, where $\tilde{\varLambda } = diag(\lambda _1, \lambda _2, \ldots \lambda _{\vert V \vert })$ is the diagonal matrix with the ordered eigenvalues as elements and $\varPhi = (\varphi _1, \varphi _2, \ldots , \varphi _{\vert V \vert })$ is the matrix with the ordered eigenvectors as columns.

2.2 Von Neumann Edge Entropy

In quantum mechanics, the density matrix is used to describe a system with the probability of pure quantum states $| \psi _i \rangle$ and each with probability $p_i$. It is defined as $\varvec{\rho } = \sum _{i=1}^V p_i | \psi _i \rangle \langle \psi _i |$. Severini et al. [7] have extended this idea to the graph domain. Specifically, they show that a density matrix for a graph or network can be obtained by scaling the combinatorial Laplacian matrix by the reciprocal of the number of nodes in the graph.

With this notation, the specified density matrix is obtained by scaling the normalized Laplacian matrix by the number of nodes, i.e. $\varvec{\rho } = \frac{\tilde{L}}{{\vert V \vert }}$. When defined in this way the density matrix is Hermitian i.e. $\varvec{\rho } = \varvec{\rho \dagger }$ and $\varvec{\rho } \ge 0, \text {Tr} \varvec{\rho } = 1$. This interpretation opens up the possibility of characterising a graph using the von Neumann entropy from quantum information theory [7]. Therefore, the von Neumann entropy is given in terms of the eigenvalues $\lambda _1$, ....., $\lambda _{\vert V \vert }$ of the density matrix $\varvec{\rho }$,

$$\begin{aligned} S_{_{VN}} = - \text {Tr}(\varvec{\rho } \log \varvec{\rho }) = - \sum _{i=1}^{{\vert V \vert }}\frac{\lambda _i}{\vert V \vert } \log \frac{\lambda _i}{\vert V \vert } \end{aligned}$$

(1)

In fact, Han et al. [8] have shown how to approximate the calculation of von Neumann entropy in terms of simple degree statistics. Their approximation allows the cubic complexity of computing the von Neumann entropy to be reduced to one of quadratic complexity using simple edge degree statistics, i.e.

$$\begin{aligned} S_{_{VN}} = 1 - \frac{1}{{\vert V \vert }} - \frac{1}{{\vert V \vert }^2}\sum _{(u,v)\in E}\frac{1}{d_u d_v} \end{aligned}$$

(2)

Therefore, the edge entropy decomposition is given as

$$\begin{aligned} S_{_{VN}}^{^{edge}}(u,v) = \frac{1}{{\vert E \vert }} - \frac{1}{{\vert V \vert }{\vert E \vert }} - \frac{1}{{\vert E \vert } {\vert V \vert }^2}\frac{1}{d_u d_v} \end{aligned}$$

(3)

where $S_{_{VN}} = \sum _{(u,v)\in E} S_{_{VN}}^{^{edge}}(u,v)$. This expression decomposes the global parameter of von Neumann entropy on each edge with the relation to the degrees from the connection of two vertexes.

3 Thermodynamic Statistics and Global Entropy Decomposition

The concept of von Neumann entropy arises in the quantum domain. Here, we commence from the Hamiltonian operator in statistical mechanics to develop thermodynamic entropy. We then decompose or project the global entropy onto edges using the eigenvectors of the normalised Laplacian matrix.

3.1 Thermodynamic Entropy

To connect the normalised Laplacian matrix to statistical mechanics, we view the eigenvalues of the Laplacian matrix as the energy eigenstates of a system in contact with a heat reservoir. These determine the Hamiltonian and hence the relevant Schrödinger equation which governs the particles in the system [3, 6]. The particles occupy the energy states of the Hamiltonian subject to thermal agitation by the heat bath [12]. The number of particles in each energy state is determined by the temperature, the assumed model of occupation statistics and the relevant chemical potential.

We consider the network as a thermodynamic system of N particles with energy states given by normalised Laplacian matrix $\tilde{L}$, which is immersed in a heat bath with temperature T. The ensemble is represented by a partition function $Z(\beta , N)$, where $\beta$ is inverse of temperature T. When specified in this way, the thermodynamic entropy is given by,

$$\begin{aligned} S = k_B \left[ \frac{\partial }{\partial T} T \log Z \right] _N \end{aligned}$$

(4)

The statistical properties of particles in the network are determined by the partition functions associated with different energy level occupation statistics. In this way, thermodynamic quantities, such as entropy, can characterise the network structure.

3.2 Maxwell-Boltzmann Statistics

The Maxwell-Boltzmann distribution relates the microscopic properties of particles to the macroscopic thermodynamic properties of matter [10]. It applies to systems consisting of a fixed number of weakly interacting distinguishable particles. These particles occupy the energy levels associated with a Hamiltonian and in our case the Hamiltonian of the network, which is in contact with a thermal bath [6].

Taking the Hamiltonian to be the normalized Laplacian of the network, the canonical partition function for Maxwell-Boltzmann occupation statistics of the energy levels is

$$\begin{aligned} Z_{_{MB}} = \text {Tr} \biggl [ \exp (-\beta \tilde{L})^N \biggr ] \end{aligned}$$

(5)

where $\beta =1/k_B T$ is the reciprocal of the temperature T with $k_B$ as the Boltzmann constant; N is the total number of particles and $\lambda _i$ denotes the microscopic energy of system at each microstate i with energy $\lambda _i$. Derived from Eq. (4), the entropy of the system with N particles is

$$\begin{aligned} S_{_{MB}} = \log Z - \beta \frac{\partial \log Z}{\partial \beta } = -N \text {Tr} \biggl \{ \frac{\exp (- \beta \tilde{L} )}{\text {Tr} [\exp (- \beta \tilde{L} )]}\log \frac{ \exp (- \beta \tilde{L} )}{ \text {Tr} [\exp (- \beta \tilde{L} )]} \biggr \} \end{aligned}$$

For a single particle, the density matrix is

$$\begin{aligned} \varvec{\rho }_{_{MB}} = \frac{\exp (-\beta \tilde{L})}{\text {Tr}[\exp (-\beta \tilde{L})]} \end{aligned}$$

(6)

Since the density matrix commutes with the Hamiltonian operator, we have $\partial \rho /\partial t=0$ and the system can be viewed as in equilibrium. So the entropy in the Maxwell-Boltzmann system is simply N times the von Neumann entropy of a single particle, as we might expect.

3.3 Edge Entropy Decomposition

Our goal is to project the global network entropy onto the edges of the network. In matrix form for Maxwell-Boltzmann statistics in Eq. (6), the entropy can be written as,

$$\begin{aligned} S_{_{MB}} = -\text {Tr} \left[ \varvec{\rho }_{_{MB}} \log \varvec{\rho }_{_{MB}} \right] = -\text {Tr} [\mathbf {\Sigma }_{_{MB}}] \end{aligned}$$

(7)

Since the spectral decomposition of the normalized Laplacian matrix is

$$\begin{aligned} \tilde{L} = \varPhi \tilde{\varLambda } \varPhi ^T \end{aligned}$$

(8)

We can decompose the matrix $\mathbf {\Sigma }_{_{MB}}$ as follows

$$\begin{aligned} \mathbf {\Sigma }_{_{MB}} = \varPhi \sigma _{_{MB}}(\tilde{\varLambda }) \varPhi ^T \end{aligned}$$

(9)

where

$$\begin{aligned} \sigma _{_{MB}}(\lambda _i) = -N {\frac{e^{-\beta \lambda _i}}{\sum _{i=1}^{\vert V \vert } {e^{-\beta \lambda _i}}} \log {\frac{e^{-\beta \lambda _i}}{\sum _{i=1}^{\vert V \vert } {e^{-\beta \lambda _i}}}}} \end{aligned}$$

(10)

As a result, we can perform edge entropy projection of the Maxwell-Boltzmann statistical model using the Laplacian eigenvectors [11]. The result of the entropy for each edge (uv) is given as,

$$\begin{aligned} S_{_{MB}}^{^{edge}}(u,v) = \sum _{i=1}^{{\vert V \vert }} \sigma _{_{MB}}(\lambda _i) \varphi _i \varphi _i^T \end{aligned}$$

(11)

Thus, the global entropy can be projected on the edges of the network system. This provides useful measures for local entropic characterisation of network structure in a relatively straightforward manner.

4 Experiments and Evaluations

In this section, we describe the application of the above methods to the analysis of interregional connectivity structure for fMRI activation networks for normal and Alzheimer’s patients. We first examine the dependence of the edge entropy components on node degree and temperature and compare their performance with von Neumann entropy. Then we apply edge entropy-based analysis to distinguish between different stages in the development of Alzheimer’s disease, and fMRI data for normal subjects. We explore whether we can identify specific inter-regional connections and regions in the brain associated with the neurodegeneration caused by the onset of Alzheimer’s disease. To simplify the calculations, the Boltzmann constant is set to unity in our experiments.

4.1 Dataset

The fMRI data were obtained from the ADNI initiative [9]. fMRI images of subjects brains were taken every two seconds and are used to compute the Blood-Oxygenation-Level-Dependent (BOLD) signals for different anatomical brain regions. To do this the fMRI voxels were aggregated into larger regions of interest (ROIs). The different ROIs correspond to different anatomical regions of the brain and are assigned anatomical labels to distinguish them. There are 90 such anatomical regions in each fMRI image. The correlation between the average time series in different ROIs represents the degree of functional connectivity between regions which are driven by neural activities.

We construct a graph to represent the pattern of activities using the cross-correlation coefficients for the average time series for pairs of ROIs. We create an undirected edge between two ROI’s if the cross-correlation coefficient between the time series is in the top 40% of the cumulative distribution. This cross-correlation threshold is fixed over all of the available data, which provides an optimistic bias for constructing graphs. Those ROIs that have missing time series data are discarded. Subjects fall into different categories according to the degree of severity of the disease, there are normal subjects, those with early mild cognitive impairment, those with late mild cognitive impairment and those with full Alzheimer’s. The data supplied 705 patients, including 105 subjects with Alzheimer’s disease (AD), 193 normal healthy control subjects (NC), 240 in the Early Mild cognitive impairment (EMCI) and 167 in the Late Mild cognitive impairment (LMCI).

Fig. 1.

Scatter plot of thermodynamic edge entropy compared to the von Neumann entropy with different value of temperatures.

4.2 Experimental Results

We first investigate the relationship between the mean edge entropy computed using Maxwell-Boltzmann statistics and von Neumann entropy. Figure 1 shows the edge entropy with varying temperatures. The statistical entropy exhibits a transition in behaviour with respect to the von Neumann entropy with varying temperature.

For example, at the high temperature ($\beta = 0.1$), the thermodynamic entropy is roughly in linear proportion to the von Neumann entropy. As the temperature reduces, they take on an approximately exponential dependence. At low temperature, the thermodynamic edge entropies decrease monotonically with the von Neumann edge entropy ($\beta = 10$). Therefore, at high temperature, the statistical and von Neumann edge entropies are proportional, while at low temperature they are in inverse proportion.

Then, we apply the edge entropy computations to fMRI brain networks, with the aim of determining which anatomical regions play the strongest role in the development of Alzheimer’s disease. Figure 2 shows the different edge entropy distribution for the Alzheimer’s disease (AD) and healthy control (Normal) samples. Compared to the von Neumann entropy which does not show a clear difference in distributions between the two groups, the thermodynamic entropy better distinguish the detailed distribution of edge entropy. The edge entropy in the case Alzheimer’s disease tends towards lower values. This observation is more palpable in the cases of the Maxwell-Boltzmann edge entropy distributions, as shown in Fig. 2(b), with more edges tending to occupy the low entropy region.

Fig. 2.

Edge entropy distribution of fMRI networks with (a) von Neumann entropy, (b) Maxwell-Boltzmann statistics. Two groups of patients, Alzheimer’s disease (AD) and healthy control (Normal).

Next, we select the edges with the largest 10% of entropy in the anatomical regions to reduce the feature dimension. This gives 278 significant edges as a feature vector. We explore whether these feature vectors can be used to classify normal healthy subjects and patients with early Alzheimer’s disease. Figure 3(a) is the visualisation of the three-dimensional principal components for four groups using Fisher’s linear discriminant analysis (LDA). Three principal eigenvectors show the cluster of each group. The palpable feature is that the statistical edge entropy in Maxwell-Boltzmann case can give the separation among the four subject groups.

If we regard the principal components as the feature vectors on each sample graph, we can apply C-SVM (Support Vector Machine) to classify four groups. The data are manually separated into two parts as 500 samples for the training data and 205 samples for the testing data (the rest of the raw data excluding the training set). The 10-fold cross-validation with the grid search method is used to find the optimal parameters (c and g)in C-SVM with Gaussian kernel. The training and testing accuracies are shown in Table 1.

Fig. 3.

(a) Visualisation of three dimensional principal components of thermodynamic edge entropy in four groups of Alzheimer’s disease. (b) Significant differences between edge entropy associated with diseased areas in the brain. We use the standard deviation of thermodynamic entropy to identify the divergence between AD and HC groups for each edge.

Table 1. SVM Classification Accuracy. In the four group classification, 500 samples are used for training and 205 samples for testing. For AD and Normal binary classification, 200 samples are used for training and 98 samples for testing. For EMCI and Normal binary classification, 300 samples are used for training and 133 samples for testing.

Table 1 shows that the edge entropies in Maxwell-Boltzmann statistics are good features to identify Alzheimer’s disease. For all the groups of patients, the total classification accuracy can reach 87.80%. In term of the binary classification between Early Mild cognitive impairment (EMCI) and healthy control (Normal), the thermodynamic edge entropy presents a better performance to classify the early disease which is helpful for clinical application. Thus, we can apply the resulting method to identify fMRI activation networks from patients with suspected Alzheimer’s disease.

On the other hand, identifying diseased regions in the brain is also important. Several studies have shown that different anatomical structures can be analysed using the properties of the corresponding ROIs, and are important for understanding brain disorders [2, 4]. Here, we use the difference in standard deviation for the thermodynamic entropy to identify the sources of significant variance between AD and HC groups. Figure 3(b) plots the greatest variance of edge entropy for different anatomical regions (edges). The entropic measurements in the brain areas, such as the Paracingulate Gyrus, Parahippocampal Gyrus, Inferior Temporal Gyrus and Temporal Fusiform Cortex, suggest that subjects with AD experience loss of interconnection between these regions in their brain network during the progression of the disease.

Table 2. Top 10 ROIs with the most significant difference in edge entropy between the Alzheimer’s disease (AD) and Health Control (Normal) groups.

As listed in Table 2, the ten anatomical regions with the largest entropy differences for subjects with the full AD are Paracingulate Gyrus, Parahippocampal Gyrus, Temporal Fusiform Cortex, etc. This result is consistent with the previous study reported in [4, 5]. For example, the parahippocampal gyrus has consistently been reported as being vulnerable to pathological changes in Alzheimer’s disease (AD), which is closely related to entorhinal and perirhinal subdivisions as the most heavily damaged cortical areas for the disease [13]. The Frontal Medial Cortex and Temporal Fusiform Cortex are memory-related cognitive areas. They are severely damaged by Alzheimer’s disease and affect recognition memory for faces. Overall, the loss of connection between these brain regions results in significant functional impairment between healthy subjects and patients with the AD.

In conclusion, both statistical methods and von Neumann edge entropies can be used to represent changes in network structure. Compared to the von Neumann edge entropy, thermodynamic edge entropies are more sensitive to sample variance associated with the degree distribution. Maxwell-Boltzmann statistics reflect strong community structure which is more suitable for representing a detailed structure of the degree distribution.

5 Conclusion

In this paper, we show how to decompose the global network entropies resulting from thermodynamic occupation statistics onto the constituent edges of a graph. We refer to the resulting statistical quantities as Maxwell-Boltzmann edge-entropies. The method uses the normalised Laplacian matrix as the Hamiltonian operator of the network to compute the corresponding partition functions. We undertake experiments to analyse the thermodynamic edge entropies and compare them to their von Neumann counterparts. Experiments reveal that the Maxwell-Boltzmann edge entropy distributions can effectively in characterising detailed variations in the network structure. It outperforms the von Neumann entropy in this respect. Finally, we apply this novel method to provide insights into the neuropathology of Alzheimer’s disease. The thermodynamic edge entropy distribution is capable of discriminating between subjects suffering from Alzheimer’s and healthy subjects.