Multispectral MRI image segmentation using Markov random field model

Abstract

Magnetic resonance imaging (MRI) is used to capture images in different modalities such as T1-weighted, T2-weighted, and PD-weighted. This paper proposes a new method for the fusion of different channels in MRI image segmentation. In the reported work, a new feature vector for multispectral MRI brain segmentation is proposed. Fuzzy C-means clustering method is applied on the three different extracted feature vectors, and results are reported. Experimental results show that the proposed feature vector presents good noise immunity. Paper reports a new segmentation method based on Markov random field and the proposed feature vector to combine spatial and spectral information for MRI image segmentation. The proposed method was applied on the BrainWeb MRI image dataset with added noise, and the segmentation results are reported and compared with some known reported works.

1 Introduction

Magnetic resonance imaging (MRI) is a powerful and popular tool for medical imaging. MRI is used in different treatment procedures such as diagnosis and follow-up. MRI is an effective diagnostic tool in many medical treatments, in which, image segmentation plays an important role [1]. Manual segmentation of brain is often time-consuming; therefore, automatic segmentation can be used to speedup the process. MRI images from brain are segmented into three classes such as gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) [2]. Structure of the human brain is very deformable and has complex boundaries between different regions.

Moreover, MRI images have common artifacts such as partial volume effect (PVE) and intensity non-uniformity (INU) that are resulted during the imaging process. In MRI imaging, each voxel represents a combination of different types of tissues within a 3D block of a brain slice. PVE is a representation of the complexity of the combination of different types of tissues within a voxel. High PVE in MRI imaging makes the image segmentation more difficult. INU is caused due to the limitations in hardware and degrades the quality of acquired MRI images [3, 4].

Finite mixture model (FMM) is commonly used for MRI brain segmentation. Greenspan et al. [5] proposed constrained Gaussian mixture model (CGMM) for MRI image segmentation where a large number of Gaussian models are used to model each tissue. Tohka et al. [6] used real-coded genetic algorithm (RCGA) to estimate mixture parameters for MRI image segmentation. Expectation maximization (EM) algorithm is used for parameter approximation. However, this method has a high computational cost and can fall into local maxima. In another reported work, Tohka et al. [7] proposed a combination of EM and GA algorithms to estimate parameters for MRI image segmentation.

In image segmentation problem areas, such as, remote sensing and MRI medical imaging, Markov random field (MRF) is reportedly used for the fusion of spectral, spatial, and contextual properties [8, 9]. Van Leemput et al. [9] proposed one of the first and most reliable methods for MRI brain segmentation called Koen Van Leemput (KVL), which is based on MRF algorithm. KVL uses EM algorithm for parameter approximation. The conventional MRF uses EM and simulated annealing methods both with high computational costs [2, 10]. Yousefi et al. [2] used a combination of ant colony optimization (ACO) algorithm and gossiping algorithm for the optimization stage in the MRF model. Sahar et al. claim that the reported algorithm outperforms the conventional MRF-simulated annealing and MRF–ACO algorithms and can operate in real time. In another reported work, a combination of MRF and a typical clustering method mainly was used to increase the robustness of the clustering method against noise [11]. Rajapakse et al. [12] used Gaussian mixture model (GMM) for brain segmentation. In their work, MRF model was used to improve the segmentation result. Marroquín et al. [13] proposed another method for MRI brain segmentation. This algorithm uses maximum a posteriori (MAP) estimator and maximization of the posterior marginals (MPM) estimator where a variant of EM algorithm for robust approximation of MAP is defined for MRF. This algorithm is called MPM–MAP. Zhang et al. [14] proposed another related method based on hidden Markov random field (HMRF) model that uses EM algorithm to fit the HMRF parameters and is called EM-HMRF.

Different extensions of fuzzy C-means (FCM) algorithm are proposed for MRI brain image segmentation [15, 16]. Rivest-Hénault and Cheriet [17] proposed a level set method for MRI brain segmentation. Level set method needs good initialization to prevent it from falling into local minima. They used FCM for the initialization and then applied level set method to improve the results. Wu et al. [18] proposed a method to combine support vector machine (SVM) and MRF methods for MRI image segmentation. They proposed a new energy function for MRF method based on SVM classification. Their proposed method suffers from high execution time for the training and validation of the results.

Atlas-based methods are active research areas in MRI image segmentation. Atlas-based methods use different pre-labeled images and prior anatomical information for brain segmentation. These methods consist of three main steps such as registration, label propagation, and final segmentation. Atlas-based methods are also proposed for brain segmentation [19].

This paper proposes a new feature vector for multispectral MRI image segmentation with high noise immunity. The use of T1-weighted image for MRI image segmentation of the brain is well studied. Images, such as T2-weighted and Proton Density-weighted (PD-weighted) that are produced by MRI, can be used as additional information to increase the segmentation accuracy. In the proposed approach, using the three aforementioned MRI imagery products, a wavelet-based method is applied to extract a feature vector. The extracted feature vector is named modality fusion vector (MFV). Later on, the MFV is used in FCM and a combination of FCM and MRF methods (FCMRF) for segmentation. In the reported work, BrainWeb dataset is used for the experiments.

2 Discrete wavelet transform

Discrete wavelet transform (DWT) is mathematical model for multi-resolution analysis. Wavelet transform is a common method for spatial frequency analysis. 1D continuous wavelet transform (CWT) is represented by Eq. 1.

$$\begin{aligned} \hbox {CW}\left( {a,b} \right) =\frac{1}{\sqrt{a}}\int y\left( t \right) \varphi \left( {\frac{t-b}{a}} \right) \end{aligned}$$

(1)

where $y(t)$ is the input signal, $\varphi$ is the wavelet function. Parameters $a$ and $b$ are scaling and translation parameters, respectively. The base functions of a DWT are captured by means of sampling from CWT. ${\varphi }_{j,k}(t)$ function is presented in Eq. 2.

$$\begin{aligned} \varphi _{j,k} \left( t \right) =2^{{-n}/{2}}\varphi \left( {2^{-n}t-m}\right) ,m,n\in N \end{aligned}$$

(2)

2D DWT is derived from 1D wavelet transform by applying filters in rows and columns. 2D DWT divides the images into four different sub-bands i.e., LL, LH, HL and HH. Each sub-band can represent an image and they approximate vertical, horizontal, diagonal details, respectively.

3 Proposed method

This section presents a method for multispectral MRI image segmentation. In the first stage, a wavelet-based data fusion method is proposed and MFV is extracted. In the second stage, MFV and two different feature vectors are applied on FCM and different label fields are produced. The result shows that MFV has high noise immunity compared with other feature vectors. In the third stage, the feature vector extracted from T1-weighted image is used for the initial segmentation. Later on, a MFV-based energy function for the MRF optimization is proposed that is called extended FCMRF (E-FCMRF). This energy function uses MFV-based label field to incorporate the spectral features in segmentation of MRI image. This energy function is compared against another energy function that uses T2-weighted and PD-weighted label field to incorporate the spectral features in segmentation and results are reported.

3.1 Modality fusion vector extraction

The MFV is extracted from the fusion of three T1-weighted, T2-weighted and PD-weighted images. Wavelet analysis is used for data fusion and extraction of the MFV. T1-weighted images have higher resolution than T2-weighted and PD-weighted images. Therefore, these images are more detailed than other spectral images. This is why in many reported works only T1-weighted images are used for MRI brain image segmentation. This paper proposes fusion of different modalities to improve segmentation accuracy for MRI brain images in high level of noise. Construction of MFV includes two stages. Initially, different image modalities are divided into four different channels such as Low horizontal and low vertical (LL) and the detailed images consist of high horizontal and low vertical (HL), low horizontal and high vertical (LH), and high horizontal and high vertical (HH) frequencies. In the second stage, T2-weighted and PD-weighted images are used to produce an approximated image. Wavelet fusion method [20, 21] is described in Fig. 1. Multispectral MRI images are not taken simultaneously like different multispectral images such as in remote sensing applications. Therefore, different modalities need a registration step to geometrically match the channels. Wavelet fusion can be applied to BrainWeb dataset due to the registration of different channels in this dataset. Otherwise for datasets that are not registered, the registration of different channels is needed.

Fig. 1

Proposed wavelet fusion method

Approximation image is made by weighted averaging of T2-weighted and PD-weighted images. Detailed images are also produced using weighted averaging of T1-weighted and T2-weighted images. The calculation method is presented in Eqs. 3 and 4.

$$\begin{aligned}&\hbox {APP}\_\hbox {img} = 0.5\times \hbox {App}\left( {T2\_weighted} \right) \nonumber \\&\qquad \qquad \qquad +\,\hbox {App}\left( {PD\_weighted} \right) \end{aligned}$$

(3)

$$\begin{aligned}&\hbox {Detail}\_\hbox {img} = 0.5\times \hbox {Detail}\left( {T1\_weighted} \right) \nonumber \\&\qquad \qquad \qquad +\,\hbox {Detail}\left( {T2\_weighted} \right) \end{aligned}$$

(4)

$F1$ is a one-dimensional feature vector and only T1-weighted image is used for segmentation. $F2$ is a three-dimensional feature vector. $F1$ and $F2$ are presented in Eqs. 5 and 6, respectively.

$$\begin{aligned} F1&= \left[ {T1\_{weighted}(i,j)} \right] ^{T}\end{aligned}$$

(5)

$$\begin{aligned} F2&= \left[ {{\begin{array}{l} {T1\_{weighted}(i,j)} \\ {T2\_{weighted}(i,j)} \\ {PD\_{weighted}(i,j)} \\ \end{array} }} \right] \end{aligned}$$

(6)

$i,\, j$ are voxels location in image. MFV is a two-dimensional feature vector. MFV improves the segmentation capability of the FCM method on multispectral MRI images. The MFV is presented in Eq. 8.

$$\begin{aligned}&\hbox {WF}=\hbox {Wav}\_\hbox {Fusion}(T1\_{weighted},\;T2\_{weighted},\nonumber \\&\qquad \quad PD\_{weighted})\end{aligned}$$

(7)

$$\begin{aligned}&\hbox {MFV}=\left[ {T1\_{weighted}(i,\,j),\,\hbox {WF}} \right] ^{T} \end{aligned}$$

(8)

These three different feature vectors i.e., $F1,\, F2$ and MFV, will be used in FCM method for multispectral MRI segmentation.

3.2 Combination of FCM clustering method and MRF model

Clustering methods such as finite mixture models (FMM) and FCM algorithms are commonly applied on brain image segmentation [12, 17]. FCM algorithm is an iterative algorithm. Equation 9 is the objective function for FCM that has to be minimized.

$$\begin{aligned} J_m =\sum \limits _{i=1}^N J_i =\sum \limits _{i=1}^N \left( {\sum \limits _{j=1}^C u_{ij}^m \Vert x_i -c_j\Vert ^{2}} \right) ,\;m\in R,m>1\nonumber \\ \end{aligned}$$

(9)

where $N$ is number of samples, $x$ is n-dimensional sample and $C$ is number of classes. $c_{j}$ is the n-dimensional center of the cluster $j$. Equations 10 and 11 show iterative steps to be followed to update different parameters such as center of the clusters and the degree of the class membership.

$$\begin{aligned} u_{ij}&= 1\bigg /\sum \limits _{k=1}^C \left( {\frac{\Vert x_i -c_j\Vert }{\Vert x_i -c_k\Vert }} \right) ^{\frac{2}{m-1}}\end{aligned}$$

(10)

$$\begin{aligned} c_j&= \sum \limits _{i=1}^N u_{ij}^m \times x_i \bigg /\sum \limits _{i=1}^N u_{ij}^m \end{aligned}$$

(11)

where k is iteration number. Iteration stops when $\max _{ij} \left\{ {\left| {u_{ij}^k -u_{ij}^{k-1} } \right| } \right\} <\varepsilon$, where $\varepsilon$ is a termination criterion between 0 and 1. Figure 2 shows a flow chart for the iterative process used for updating $u_{ij}$ and $c_{j}$. MRF model is proposed by Besag [22]; however, this method was initially applied to the image processing area by Geman and Geman [23]. In this method, segmentation problem is turned into a labeling problem and goal is to find the best label field for all the segments. This method uses an energy function together with an optimization algorithm to minimize the energy function. This method uses EM for the parameter estimation and the simulated annealing to optimize the energy function.

Fig. 2

Flow chart for the iterative process of calculating the degree of the class membership $(u_{ij})$ and center $(c_{j})$

MRF is one of the best methods for contextual image segmentation [8]. This method is applied on the neighborhood information in image segmentation. Conventional MRF method is very time-consuming. Combination of MRF and a typical clustering method was used to increase the robustness of the clustering method against the noise and reduce execution time for the MRF. Rajapakse et al. [12] used GMM for initial brain segmentation. Later on, MRF method was used to smooth different segments and to make the GMM more robust against noise.

Dubes et al. [11] also proposed a combination of clustering methods together with the MRF method to reduce computation burden and to increase robustness against noise. They used K-means clustering for the initial segmentation followed by the MRF method to increase the number of eliminated fragments in different segments. This method is proposed for one-dimensional image segmentation and Eq. 12 shows its energy function.

This energy functions consists of two parts, the first part includes cluster binding and the second part calculates neighboring energy for all the cliques in the image. The proposed method uses only first part of the energy function in implementing the MRF method. $E(y)$ is energy function used to incorporate spatial information in image segmentation for label field [11, 12].

$$\begin{aligned} U(y)=\frac{1}{2}\ln \left( {\sigma _t^2 } \right) +\frac{\left( {x-\mu _t } \right) ^{2}}{2\sigma _t^2 }+E(y) \end{aligned}$$

(12)

where $U(y)$ is total energy of label field $y$. $\mu _t$ and $\sigma _t$ are mean and variance of segment $t$, respectively. Variable $x$ stands for the pixel intensity.

3.3 Proposed method for multispectral MRI image segmentation

In this paper, two different methods are compared against each other. Each method follows a different approach in using spectral features in MRF. In the first method, different channels are segmented using FCM method. Here, a fuzzy segmentation is produced, and a membership degree is assigned to each pixel representing the membership of each pixel within a cluster. The highest class membership is used to select the class of each pixel.

Therefore, three different label fields are produced including $T1$ label field, $T2$ label field, and PD label field. The $T1$ label field is used for the initial segmentation, and the other two label fields are used to incorporate spectral information in MRI segmentation. Neighborhood energy for this labeling is presented with $V_{\text {in-plane}}$ ($T1$ label field) and the other two label field as $V_{\text {out-plane}}$. Then, an energy function such as Eq. 12 is used in simulated annealing method for the optimization step. In this paper, $E(y)$ is calculated using Eqs. 13 and 14.

$$\begin{aligned}&E\left( y \right) =\sum \limits _{c\in C} \left( \sum \limits _{s=1}^8 V_{\text {in-plan}} \left( {i,j} \right) +\,0.5\right. \nonumber \\&\qquad \qquad \left. \times \sum \limits _{s=1}^2 V_{\text {out-plan}} \left( {i,k} \right) \right) ,\quad i,\,j,\,k\in S\end{aligned}$$

(13)

$$\begin{aligned}&V_{\text {in-plan}} =V_{\text {out-plan}} =\left\{ \begin{array}{lll} -0.5\,&{}\quad \hbox {if}&{}\quad y_{i} =y_j \\ +0.5\,&{}\quad \hbox {if}&{}\quad y_{i} \ne y_j \\ \end{array} \right. \!,\quad i,\, j\,\in S\nonumber \\ \end{aligned}$$

(14)

where C presents the all possible cliques in the image and $S$ is the lattice defined on the image. This work applies a new energy function for multispectral MRI image segmentation. In the proposed method, F1 and MFV vectors are segmented using FCM method and, consequently, two different label fields are produced. Like previous method T1-weighted label field is used as initial segmentation. Another label field that is produced by the MFV is used to incorporate spectral information in MRI image segmentation. According to Table 1, this label field has higher accuracy than other label fields such as PD and $T2$ label fields. Therefore, the value of this label field has significant effect on the energy function. Neighborhood energy for $T1$ labeling is presented by $\hbox {V}_{\text {in-plane}}$ and the other proposed label field by $\hbox {V}_{\text {out-plane}}$. In the proposed method, $E(y)$ is presented by Eqs. 15 and 16 presents $\hbox {V}_{\text {in-plane}}$ and $\hbox {V}_{\text {out-plane}}$ values for different pixel locations.

$$\begin{aligned}&E\left( y \right) =\sum \limits _{c\in C} \left( 0.5\,\times \left[ {\sum \limits _{s=1}^8 V_{\text {in-plan}} (i,j)} \right] \right. \nonumber \\&\qquad \quad \,\left. +\,0.5\times V_{\text {out-plan}} \left( {i,k} \right) \right) ,\,i,\,j,\, k\,\in S\end{aligned}$$

(15)

$$\begin{aligned}&V_{\text {in-plan}} =V_{\text {out-plan}} =\left\{ \begin{array}{lll} -1&{}\quad \mathrm{if}&{}\quad y_i =y_j \\ +1&{}\quad \mathrm{if}&{} \quad y_i \ne y_j \\ \end{array} \right. \;i,\,j\,\in S \end{aligned}$$

(16)

Flow chart of the proposed method for combining FCM with MFV and MRF method is presented in Fig. 3.

Fig. 3

Flow chart of the proposed method for multispectral MRI image with high level of noise

Table 1 Dice index for different noise levels and three different INU levels

4 Experimental results

In the reported work, a well-known BrainWeb dataset that is captured at McConnell Brain Imaging Centre of the Montreal Neurological Institute, McGill University [24], was used to evaluate the proposed method. Dice coefficient is used for comparison different feature vectors. Dice coefficient is described by Eq. 17.

$$\begin{aligned} D\left( k \right) =\frac{2\,\times V_{\mathrm{p} \cap \mathrm{g}} (k)}{V_\mathrm{p} (k)+V_\mathrm{g} (k)} \end{aligned}$$

(17)

In this equation, $V_\mathrm{p} (k)$ represents the proposed method of labeling for class $k,\, V_\mathrm{g} (k)$ represents the ground truth labeling for class $k$ and $V_{\mathrm{p}\cap {\mathrm{g}}} (k)$ is the number of pixels with the same label in the proposed method and ground truth for class $k$. In other words, dice coefficient represents the overlap results of the aforementioned methods for the class $k$.

4.1 Experiment 1

First experiment is aimed to compare three different feature vectors presented in Sect. 3.1. Table 1 shows average value of different INUs within each noise level. This experiment is performed on different INUs such as 0, 20, and 40 % for two different high noise levels i.e., 7 and 9 %.

Table 1 shows that the $F2$ feature vector has the worst average accuracy compared with other feature vectors, while the proposed feature vector has the best average accuracy. Daubechies 6-tap wavelet is used for the MFV feature vector extraction. Difference between values of MFV and F1 for 7 % noise level is 0.09 % while difference for 9 % noise level is 2.44 %. Therefore, results show that increasing noise level will increase effectiveness of the proposed method. Results for slices with 9 % noise and 40 % INU are presented in Fig. 4.

Fig. 4

Segmentation results for two different feature vectors

4.2 Experiment 2

Experiment 2 compares two different MRF methods that are introduced in Sect. 3.2. Figure 5 depicts different segmentation results for MFV with FCM and the proposed energy function for MRF method. Figure 5 presents segmentation results for MRI images with 9 % noise level and 40 % INU. A detailed comparison between different slices is presented in Table 2. Table 2 presents dice coefficient for different segments for 9 % noise and 40 % INU.

Fig. 5

Segmentation results for the two proposed methods

Table 2 Segmentation results for some slices

Figure 6a depicts dice coefficient for different iterations calculated for the slice 80 with 9 % noise and 40 % INU using the proposed method. Figure 6b depicts averaged dice coefficient for different iterations calculated for all of the slices, with 9 % noise and 40 % INU using the proposed method. Figure 6a, b show that during early iterations, the accuracy increases quickly. However, after a while, its rate is decreased.

Fig. 6

a Accuracy in different iterations. b Averaged accuracy in different iterations

Figure 6a shows that WM has better segmentation accuracy in comparison with other segments. Figure 6b shows that WM has the lowest segmentation accuracy. This effect is due to low volume of WM in initial and final slices of the brain. This is why the WM segmentation of these slices produces a bad segmentation result. Therefore, average measure on all of the slices resulted in low segmentation accuracy in comparison with the results from each individual segment.

Using finite mixture models, researchers have reported excellent results in MRI brain segmentation. Ferreira da Silva [25] compared three effective and well-known methods such as MPM–MAP [13], KVL [9], and EM-HMRF [14] against the reported method that is called Dirichlet Process Mixture Model (DPMM). Results show that MPM–MAP outperforms other methods. Hence, it was decided to use experimental results out of KVL, MPM–MAP, and GMM–GA [7] for the evaluation purpose. GMM–GA method is applied on LONI (Library of Neuro Imaging) software from university of UCLA [26], for the evaluation of the proposed method. Reason for selecting these three methods is their similar approach for segmentation, accuracy, and their popularity. Intension was to compare the proposed method against strong competitors. Experimental results show that applying the proposed method for multispectral image segmentation in two different high noise levels will produce better accuracy compared with FCMRF and GMM–GA. Experimental results are compared versus two well-known and effective methods such as KVL and MPM–MAP. Detection accuracies for seven different methods are presented in two different noise levels of 7 and 9 % in Table 3.

Table 3 Dice index for different noise levels and three different INU levels for proposed MRF method

In Table 3, weighted average of other three columns i.e., CSF, GM, and WM is reported in a column labeled “Average”. Each one of the CSF, GM, and WM values are corresponding average values on three INU levels i.e., 0, 20, and 40. Comparing the proposed method against KVL, the proposed method provides better weighted average accuracy. The accuracy of the proposed method is less than MPM–MAP; however, their difference is small.

5 Conclusions and future work

This paper proposes a new data fusion method for multispectral MRI image segmentation on noisy MRI images.

Different feature vectors are used for comparison in high noise levels. FCM method is applied on different feature vectors. The reported results show that MFV produces promising result against noise. This paper also proposes a new method based on the use of a combination of FCM and MRF methods. MFV is used to incorporate spectral information in the optimization step. Results show that the proposed method has a good immunity against noise and different artifacts such as INU. Results also show that in comparison with the other well-known methods such as GMM–GA, FCM, and FCMRF the proposed method provides better segmentation accuracy for different tissues. The proposed method uses simulated annealing as for the optimization step; therefore, it has high computational complexity and is slow. Improvement of the execution time is a research necessity that is left for the future work.