A Nonlocal Based Two-Step Method Applied to MRI Brain Image Segmentation

Abstract

Accurate brain image segmentation is a challenge and meaningful task that assists physicians in the disease diagnosis. In this paper, we present a nonlocal based two-step method for image segmentation. First step is to denoise the MRI brain image with adaptive nonlocal regularization. The second step is our new nonlocal based regularized segmentation. We force the result segmentation of grey matter(GM), white matter(WM) and cerebrospinal fluid(CSF) keeping as much structure as possible by using nonlocal regularization, which has significant meaning in diagnosis. With experiments on synthetic images from BrainWeb and real MRI images from Zhejiang Cancer Hospital, we find that our method performances very well on both databases.

Z. Chen—This is the corresponding author. This work was partly supported by National Natural Science Foundation of China 11426205 and Research Fund Projection of Zhejiang Medical School 2013ZR04.

1 Introduction

Image segmentation is a progress of extracting significant features or regions from images, which plays an important role in medical image analysis [8]. The intensity of pixels, contour, color and texture can be used as the segmentation features. Its purpose is to divide image into several disjoint regions, and each region is consistent, while the characters to adjacent areas have obvious differences. Much significant information can be gotten from accurate segmented images, which facilitates computer aided disease diagnosis [9]. Though there are pretty many works on MRI brain segmentation [5], it still needs further research based on real problems.

Bias field, also known as intensity inhomogeneity, usually manifests itself as a smooth spatially varying function in MR brain images. It alters image intensities to be not constant for the same tissue, even for pixels from same tissue nearby. Numerous approaches have been proposed to correct the intensity inhomogeneity, one can refer to [11] for detailed methods. Partial volume(PV) phenomenon is another factor which is often seen in MR brain images, it is particularly notable at the junction of different tissues. Furthermore, noise is another problem we meet in dealing with MR brain image, while computer is sensitive to the spurious intensity variations. Bias field, PV and noise together make MRI brain image segmentation a thorny issue. What’s more, MR brain images are rich in structures which plays an important role in disease analysis and diagnosis. In the last two decades, researchers proposed a lot of methods to get an accurate segmentation. In [10], Li et al. proposed a coherent local intensity clustering (CLIC) criterion for segmenting images with intensity inhomogeneity. But this does not work well in texture-rich regions. In [2], Caldairou et al. proposed a method which integrated nonlocal regularization into FCM segmentation, we denote it as $NL\_R\_FCM$. Results show that it is good at preserving brain structures while it loses basic assumption about the smoothness of bias field. But noise in the MR brain image makes above methods less effect, especially for heavy noise. Z.Chen [3] et.al propose a method that takes all the above impact factors into account, we denote it as $NL\_R\_CLIC$ hereafter. But in their functional, there are so many parameters which need to be adjusted. It is a hard and time consuming work, if one or several parameters are not so suitable, the results will be not so good. One can refer to [6] for more works on nonlocal way segmentation.

In this paper, we propose a nonlocal based two-step method applied to MRI brain image segmentation. For the first step, we do the nonlocal denoising to reduce noise while keep more structure, get a denoised image I from original brain image f. For the second step, we proposed a nonlocal regularized functional for segmentation. With the aid of nonlocal term, we keep the structure as much as possible, which will be convenient for diagnosis. By decomposing an integrated functional into two parts, we are easier to adjust the parameters and we will get more accurate segmentation.

This paper is organized as follows. In Sect. 2, we describe proposed two-step method on image segmentation. In Sect. 3, some applications to MRI brain image segmentation are given. Section 4 includes some remarks and the next phase of research direction will be discussed.

2 Proposed Model

In this section, we will briefly introduce the basic assumptions, then present our proposed nonlocal denoise and brain image segmentation method.

2.1 Bias Field Model and Some Basic Assumptions

The bias field in a brain MR image can be modeled as a multiplicative component of an observed image, which can be expressed as follows [10]

$$\begin{aligned} f = B\times C + n, \end{aligned}$$

(1)

where f is the observed noise image, B is an unknown bias field, C is the true piecewise constant image we want to be estimated, and n is supposed to be additive gaussian noise.

To be specific, we assume that there are N types of tissues in the image domain $\varOmega$, and these tissues are located in N disjoint subregion $\varOmega _{1},\varOmega _{2},...,\varOmega _{N}$. In view of the PV effects, it is more feasible to identify tissues by seeking membership functions $U=\{u_k(x)\}_{k=1}^N$ of the subregions $\{\varOmega _k\}_{k=1}^N$, where $u_k(x)$ means the probability of the pixel x belonging to the kth class. So $0\le u_k(x)\le 1$ for $k=1,...,N$ and $\sum _{k=1}^N u_k(x)=1,\forall x\in \varOmega .$

Also we have additional assumption for equation (1). The bias field is slowly varying over the entire image domain. That is, when we consider a small local neighborhood, B almost keeps the same intensity value, i.e. $B(x)\approx B(y), \forall x \in ~O_y$.

2.2 The Proposed Method

In the chapter, we detail the nonlocal based two-step method for MR brain image segmentation.

The First Step-Nonlocal Denoise. In [1], Buades et.al proposed a nonlocal algorithm for image denoising named NL-means. The nonlocal denoise problem can be summarized into following functional,

$$\begin{aligned} \min \limits _{I} NLF(I)= \int _{x\in \varOmega }(|\nabla _{NL}I(x)| + \lambda |f(x)-I(x)|^2) dx \end{aligned}$$

(2)

Here, f(x) is observed noised intensity value in the location $x\in \varOmega$. I(x) means the true intensity value in the location $x\in \varOmega$. The second term is a fidelity, which means that the denoised image should be close to original noised image. And the first term is a regularization, which means that the denoised image should be smooth in the nonlocal sense, that is, intensities in similar patch will keep the same after denoising. The weight $\lambda$ balances the influence from each part. We should pay more attention on regularization at the beginning, for I is much closer to f, so we need to set a big $\lambda$ value; As update goes on, we find that resulted I will be more smooth and apart away from original noised image f, at that time we should pay less attention on fidelity, so we need to set a smaller $\lambda$. So in implementation, we use $\lambda _{iter}$ instead of constant $\lambda$, so we should pay less attention on the second term of equation (2), in another word, $\lambda$ should get smaller. Here we use $\lambda _{iter+1} = \beta \times \lambda _{iter},$ and when $iter = 0$, we set $\lambda _0 = \lambda$.

We notice that we used a nonlocal regularization here. The NL-means not only compares the grey level in a single point but the geometrical configuration in a whole neighborhood. This fact allows a more robust comparison that neighborhood. When images with rich structures, we will keep better structures with nonlocal regularization. More details and calculations about NL-means, one can refer to [7].

The Second Step-Nonlocal Regularized Segmentation. We propose a functional which takes the advantages of CLIC [10], intensities in a local neighbor will have consistent performance, and nonlocal regularization will keep better structures. The proposed functional can be expressed as follow,

$$\begin{aligned}&\min _{U,B,C}\mathcal {J}_{NL}(U,B,C) = \int _{\varOmega }(\underbrace{\sum _{k=1}^N\int _{\mathcal {O}_x}\mathcal {K}_{\sigma }(x,y)u_k^q(y)|I(y)-B(x)c_k|^2dy}_{CLIC}\\&+\underbrace{\lambda _1\sum _{k=1}^Nu_k^q(x)\int _{y\in N_x^R}\omega _{x,y}\sum _{j\in L(k)}u_j^q(y)dy}_{Regularization} dx, \end{aligned}$$

Here, U stands for the union of $\{u_i(x),i=1,...,m,$ m is the class index, and $x\in \varOmega \}$. $N_x^R$ means a neighborhood of x with radius R, as we just consider the characters in a local mean. $\mathcal {K}_{\sigma }$ is often chosen as a truncated Gaussian kernel

$$\begin{aligned} \mathcal {K}_{\sigma }(u) = {\left\{ \begin{array}{ll} \frac{1}{\zeta }e^{-|u|^2/2{\sigma ^2}} \quad \text{ for } |u|\le \rho \ ;\\ \quad 0 \quad \quad \quad \quad \quad \text{ else }, \end{array}\right. } \end{aligned}$$

where $\zeta$ is a normalized constant and $\sigma$ controls the radial scope of the function. Which means pixel that closer to the center will affects more on the final local energy.

Fig. 1.

The flow chart of the proposed method.

Table 1. Evaluation of tissue segmentations in terms of Dice’s similarity coefficients under different noise and bias field conditions.

This functional contains two parts, the first CLIC term is a fidelity, which defines an integrated energy which composed by energy defined at a small neighborhood of each pixel. It can be seen as a form of fuzzy C-means in a small region centered at certain respective given pixel, and this holds for each pixel in the region of interest(ROI). The second term is used for regularization, which forces the membership function to be smooth in nonlocal sense. If two patches are very much alike, the assignment of the center pixel of one patch will be much alike that from the other patch. By using nonlocal regularization one more time, the resulted image will keep the structures remained getting from last step as completely as possible.

The aim is to estimate U, B and $\{c_k\}_{k=1}^N$ by minimizing $\mathcal {J}_{NL}(U,B,C)$. The energy functional with respect to the variable U or B or C is convex given the other two variables fixed. So we can alternatively update these variables. One can refer to our previous paper [3] for detailed calculation.

3 Numerical Experiments

In this section, we apply the proposed method to simulated T1-weighted brain MR images provided by the BrainWeb [4] database and real brain MR images getting from Zhejiang Cancer Hospital to verify its effectiveness.

Fig. 2.

Applications of our method to a 3 T synthetic MR image. First row shows original brain image, skull-stripped brain image, denoised image; the ground truth GM, WM and CSF are shown on the second row; the resulted GM, WM and CSF with our method are listed in the third row; In the last row, we show the estimated bias field and resulting piecewise constant image.

With given MR brain images, we firstly use a simple way to do skull stripping, removed almost all non-necessary matters/tissues.This is not needed when deal with synthetic brain images, because we have templates downloaded from BrainWeb. Then we do nonlocal denoise to get a clean image. At last we use the nonlocal regularized segmentation. The flow chart is summarized as Fig. 1.

For the weight function in nonlocal denoise and nonlocal regularized segmentation, we use different parameters because they have different aims. The iteration factor $\beta$ in first step is set to be a constant depending on Brain images, and maximum iteration number in nonlocal denoise according to time and the segmentation. The weight function can be updated under denoising, while it doesn’t need to update in doing segmentation anymore as we do nothing change to the clean image I. In step two, the number of clusters is set to 3(only WM, GM and CSF are taken into consider), radius $\rho$ is selected to be 7, $\sigma = 8$, $h=1$, patch size of denoise window is 11, while 15 for that in segmentation. Neighborhood radius for denoise is selected according to noise degree, one can select 3 for large noise, 7 for small noise, since noise will affect the similarity of patch dramatically. We can have a glance on the denoised image to choose the almost proper value. We applied our proposed method to a synthetic MR brain image gotten from BrainWeb. With GM, WM and CSF comparison between ground truth and segmentation result as shown in Fig. 2, we can see that our method achieves a reasonable and pleasant result, almost all details are kept well after segmentation in anterior cranial and cerebral locations.

Table 2. Select comparisons of different methods under different noise and bias field conditions.

In order to show a quantitative comparison between the results getting from the method proposed in this paper and the ground true classes, we showed the segmentation accuracy with different conditions of noise and bias field in Table 1. With the proposed method, we can not guarantee to remove all irrelevant structures, and it mainly affect the segment accuracy of CSF. So we do not take the accuracy of CSF into consider. The GM and WM segmentation accuracies are measured by using the average overlap metric (AOM), which is a quantitative evaluation of performance. Overlap metric is defined for a given voxel class assignment as the sum of the number of voxels that both have the class assignment in each segmentation divided by the sum of voxels where either segmentation has the class assignment. This is the same as the Tanimoto coefficient. The AOM can be expressed as follow:

$$\begin{aligned} AOM=2\times \frac{N(I\bigcap J)}{N(I) + N(J))}\times 100\,\%, \end{aligned}$$

(3)

where $N(I\bigcap J)$denotes the number of voxels that both images I and J have the class assignment, N(I)(N(J)) denotes the number of voxels where I(J) segmentation has the class assignment. This metric approaches a value of 1.0 for results that are very similar and near 0.0 when they share no similarly classified voxels.

According to Zijdenbos statement [12] that AOM indicates excellent agreement when it is above 0.7. From the results shown in Table. 1, we can see that the proposed method is feasible and robust to bias field and noise. Though the accuracy decreases when noise and bias increase, the accuracy stays above 0.85!

Fig. 3.

Applications of our method to a 3 T real MR image. First row shows original brain image, skull-stripped brain image, denoised image; second row shows the result GM, WM and CSF. Estimated piecewise-constant image with proposed method, CLIC and NLFCM is given in the last row.

Fig. 4.

Applications of our method to another 3 T real MR image. First row shows original brain image, skull-stripped brain image, denoised image; second row shows the result GM, WM and CSF. Estimated piecewise-constant image with proposed method, CLIC and NLFCM is given in the last row.

With segmentation comparison results shown in Table. 2, we find that when noise and bias degree increases, the proposed method outperforms the other three art-to-state methods both in WM and GM aspects. The performance of $NL\_R\_CLIC$ is almost the same as the proposed method, but we need to take into consider that the two-step way to segmentation is much easier and flexible than $NL\_R\_CLIC$. We also find that the performance of CLIC[10] with slighter noise and bias is better than the proposed method in average means. We thought that the main factor affecting the segmentation accuracy of CLIC is the noise degree, for there is no special denoise term in the CLIC method. While for $NL\_R\_FCM$[2], the comparison result is compatible with that shown in paper [3], and one can refer to it for more detailed comparisons.

We also apply our methods to real MRI brain images from Zhejiang Cancer Hospital. We can see from Fig. 3 that original 3T MR brain image has big noise. Noise makes the MR brain image’s boundary unclear, where white matter interlaces with grey matter, and it produces rich structures as shown in Fig. 3. This is from brain images slice-view. We can also image that cerebral cortex has a wealth of wrinkles enclose the whole brain, and it is very important to diagnosis of kinds of diseases [8, 9]. Though noise has not much affects to human eyes, but it is horrible for computer detecting. It appears almost in a mass, and it is difficult to separate from one another. But with our proposed method, we simply do nonlocal denoise to remove the noise while keep as much structure as possible, just as you see in Fig. 3(c). Then with nonlocal regularized segmentation, we get finally piecewise constant image used for diagnosis.

For another experiment on another MR brain image, we also get a nice result as shown in Fig. 4. We can see a rich structure in the left region of the Fig. 4(a), which is corresponding to human’s hindbrain, and we see a lot of twists there. With final result GM,WM and CSF, also with estimated piecewise constant brain image, together with gyri region, which is hard for many segmentations, we get a wonderful structure keeping. Both in Figs. 3 and 4, we take CLIC and NLFCM methods for comparisons. With a rough view, we find more noise in the resulting piecewise constant images than the proposed method does, especially for NLFCM. This mainly attributes to the nature of functionality without estimation of biasfield. So with heavy bias in the brain images, NLFCM performances somewhat worse. Comparing Fig. 3(h) with Fig. 4(h), we find that larger noise makes segmentation with CLIC less effective. Another point as mentioned before, total variation based method makes the result keep less structure. As you can see in right corner for Fig. 4(h), CLIC does a wrong segmentation.

Additional notes, we use morphological operators to do skull stripping. As we can see in Figs. 3(b) and 4(b), we still find some non-brain matters, which will affect the segmentation performance. In the future, we will use more accurate skull stripping method and make the segmentation more clean and accurate.

4 Conclusions

Brain image segmentation is a challenging and meaningful task. Bias field, partial volume effect and noise together make it difficult to tackle with. In this paper, we deal it in a two-step way. First, we denoise the original MR brain image in a nonlocal form. Then we integrate CLIC and nonlocal regularization to do segmentation with the denoised image from step one. With slight manual intervention, a more accurate segmentation result can be obtained. Comparing our proposed method with existing methods by experiments on synthetic images from BrainWeb and real MR brain images from Zhejiang Cancer Hospital, the advantages of two-step method can be identified. In the future, we try to find an adaptive way to do nonlocal denoise and apply the method to cancer segmentation to improve the segmentation accuracy.