IFOC: Intensity Fitting on Overlapping Cover for Image Segmentation

Abstract

Region-based image segmentation methods often require a global statistical description of the image intensities in the entire foreground and background, which however is in general not available for real world images due to the complicated intensity variation within the regions of interest. In this paper, we propose a foreground-background segmentation algorithm for images with intensity inhomogeneity in a level set framework, which exploits simple distribution of local image intensities. We assume that the intensities of the foreground and background within a small enough neighborhood are separable and can be well approximated by two constants. We call such a neighborhood an intensity separable neighborhood (ISN). Given a set of overlapping ISNs that form a cover of the entire image domain or a region of interest, we formulate image segmentation as a problem of seeking for an optimal level set function that represents the foreground and background with its positive and negative sign, and a pair of constants that approximate the local foreground and background intensities within the ISNs. This formulation is a significant extension and improvement of our previous work. The main contributions in this paper include: (1) We eliminate an intrinsic drawback in our previous work that the fitting functions are not well defined for points far away from the zero level set, which causes unstable performance of segmentation; (2) The new algorithm is more efficient than our previous algorithm due to the sparse placement of the ISNs.

C. Li—This work is supported by NSFC (Grant numbers G0561671135, G0591630311, M0501020111531005). The authors thank the anonymous reviewers for their valuable comments.

1 Introduction

Image segmentation is a well-studied area which plays an increasingly important role in quantitatively analyzing the information in medical imaging applications. In recently years, segmentation algorithms in level set framework have attracted much attention. Comparing with other segmentation methods [7], level set method can achieve sub-pixel accuracy of the object boundaries, and automatically handled topological changes, and incorporate various prior knowledge.

Yang et al. [11] embed a Markov random field (MRF) energy function to the conventional level set energy function to enhance the robustness against noise, they explored algebraic multigrid (AMG) and sparse field method (SFM) to increase the time step and reduce the computational domain. Wu et al. [10] demonstrated that the unique global minimized value of the energy function is within the interval [−1,1] of any image, so they proposed a level set formulation with a strict convex energy function. Swierczynski et al. [9] developed a mathematical formulation to jointly segment and register based on a level set formulation.

In [6], Ngo et al. proposed level set method combining with deep learning for the automated segmentation of the left ventricle. Combination of these methods brings together the advantages of both approaches, requiring small training sets and producing precise segmentation results. However, this method do not use in images with intensity inhomogeneities.

In real-world images, intensity inhomogeneity is common, which arising from the imperfections of imaging acquisition processing and unavoidably leads to many difficulties and challenge in image processing and computer vision. So far numerous level set algorithms have been proposed to segment images with intensity inhomogeneity.

In [12], inhomogeneous objects are modeled as Gaussian distributions of different means and variances, where the original image is mapped to another domain using a sliding window to defined a maximum likelihood energy function. A level set method based on local approximation of Taylor expansion proposed by Min et al. [5] to solve the nonconvex optimization problem in the context of intensity inhomogeneity. In this method, the local statistical information and the variation degree information of intensity inhomogeneity are combined into the proposed model. This method can segment intensity inhomogeneity regions, unlike other linear approximation method. However, high computational cost occurred because of the complicated local information in the formulation.

In this paper, we propose a novel foreground-background segmentation algorithm for images with intensity inhomogeneity in a level set framework. Our method is based on a number of overlapping neighborhoods that form a cover of the image domain. The basic idea is to introduce a Intensity Fitting on Overlapping Cover (IFOC) model in a variational formulation, so that the segmentation result can be achieved by seeking for an optimal level set function. Moreover, it is not sensitive to initialization in the new proposed method. Compared with the previous work [1, 3], we demonstrate that our segmentation method is competitive and presents better results both in terms of effectiveness and efficiency.

2 Intensity Fitting on Overlapping Cover

In [2], Li et al. proposed a local binary fitting (LBF) model for image segmentation based on images with intensity inhomogeneity. The LBF model is based on the assumption that the image intensities on the two sides of the object boundary can be locally approximated by two constants. More specifically, Li et al. consider the neighborhood $\mathcal{O}_y$ for every point y in the image domain $\varOmega$. If $\mathcal{O}_y$ is divided by the object boundary into two parts, then the image intensities within the two parts can be approximated by two constants $f_1(\mathbf{y})$ and $f_2(\mathbf{y})$. Therefore, they define the following local binary fitting energy

$$\begin{aligned} E_y(\phi , f_1,f_2) = \lambda _1 \int K(\mathbf{x}-\mathbf{y})|I(\mathbf{x})-f_1(\mathbf{y})|^2 H(\phi (\mathbf{x}))d\mathbf{x}&\nonumber \\ + \lambda _2 \int K(\mathbf{x}-\mathbf{y})|I(\mathbf{x})-f_2(\mathbf{y})|^2 (1-H(\phi (\mathbf{x})))d\mathbf{x}&\end{aligned}$$

(1)

where $\phi$ is the level set function. H is the Heaviside function. Thus, image segmentation is formulated as a problem of seeking for optional level set function $\phi$ and the fitting functions $f_1(\mathbf{y})$ and $f_2(\mathbf{y})$, such that the local binary fitting energy $E_y(\phi , f_1,f_2)$ is minimized for every neighborhood $\mathcal{O}_y$ for all $\mathbf{y}$ in $\varOmega$. This can be achieved by minimizing the integration of the LBF energy $E_y(\phi , f_1,f_2)$ with respect to the neighborhood center $\mathbf{y}$ over the entire image domain $\varOmega$. Therefore, a global energy $F(\phi , f_1,f_2)$ is defined as

$$\begin{aligned} F(\phi ,f_1,f_2) = \int E_y(\phi , f_1(\mathbf{y}),f_2(\mathbf{y}))d\mathbf{y}\end{aligned}$$

(2)

The choice of the kernel function K is flexible. In this paper, it is defined by

$$\begin{aligned} {K(\mathbf{u})} = \left\{ \begin{array}{ll} a,\,for\,\vert \mathbf{u}\vert \le r \\ 0,\,otherwise \end{array} \right. \end{aligned}$$

(3)

where a is a positive constant such that $\int K(\mathbf{u})=1$, and r is the radius of the neighborhood $\mathcal{O}_{u,r}$. This kernel function K is a truncated uniform function.

Let $\phi$ be a level set function defined on the image domain $\varOmega$. We define

$$\begin{aligned} \begin{aligned} \varOmega _{+}=\{\mathbf{x}:\phi (\mathbf{x})>0\} \\ \varOmega _{-}=\{\mathbf{x}:\phi (\mathbf{x})<0\} \end{aligned} \end{aligned}$$

(4)

We consider an image with a foreground and a background, denoted by $\varOmega _f$ and $\varOmega _b$, respectively. Our goal is to find a level set function $\phi$ such that the regions $\varOmega _{+}$ and $\varOmega _{-}$ match the foreground $\varOmega _f$ and background $\varOmega _b$, respectively. To achieve this goal, we formulate our method as following.

For a point $u\in \varOmega$, we consider the image intensities $I(\mathbf{x})$ in a neighborhood

$$\begin{aligned} \mathcal{O}_{u,r}=\{\mathbf{x}:|\mathbf{x}-u|<r\} \end{aligned}$$

(5)

which includes a part of foreground and background. We assume that the intensities $\varOmega _f \cap \mathcal{O}_{u,r}$ and $\varOmega _b \cap \mathcal{O}_{u,r}$ are separable and can be well approximated by the constants $f_1(u)$ and $f_2(u)$, respectively. We call such a neighborhood an intensity separable neighborhood (ISN), and u is the ISN center.

Due to the intensity inhomogeneity in the image, it does not make sense to approximate the intensities in $\varOmega _b$ and $\varOmega _f$ by two global constants $c_1$ and $c_2$, respectively. However, the local intensities $I(\mathbf{x})$ for $\mathbf{x}$ in $\mathcal{O}_{u,r}\cap \varOmega _f$ and $\mathcal{O}_{u,r}\cap \varOmega _b$ can be well approximated by the constants, denoted by $f_1(u)$ and $f_2(u)$, respectively.

The ISN center can be placed in different ways. In this paper, the ISN centers are only placed near the zero level set of the level set function, such that the ISN is always divided by the zero level set into two parts. Thus, $f_1(u)$ and $f_2(u)$ are defined as the constants that approximate the image intensities in the two parts of the ISN divided by the zero level set. Note that, in RSF model in [3], the fitting functions $f_1(u)$ and $f_2(u)$ are not well defined for u far away from the zero level set, which does not pass through the ISN centered at u. Therefore, the fitting functions $f_1$ and $f_2$ are not well defined at such location. This is a theoretical drawback of RSF model, which in fact cause some undesirable segmentation results as shown in Figs. 3 and 4, due to the not-well-defined fitting functions $f_1$ and $f_2$ in RSF model. In this paper, we eliminate this drawback by placing the ISN centers only near the zero level set, such that the ISN is always divided by the zero level set into two parts, as shown in Fig. 1.

Given a set of ISNs with centers at $u_k$, $k=1,\cdots , N$, which form an overlapping cover of the entire image domain or a region of interest, both denoted by $\varOmega$. For each ISN centered at $u_k \in \varOmega$, we define the following local intensity fitting energy:

$$\begin{aligned} E_{u_{k}}( \!\!&\phi , f_1(u_k),f_2(u_k)) \nonumber \\&=\lambda _1 \int _{\varOmega _{+}}K(\mathbf{x}-u_k)|I(x)-f_1(u_k)|^2d\mathbf{x}\nonumber \\&+\lambda _2 \int _{\varOmega _{-}}K(\mathbf{x}-u_k)|I(x)-f_2(u_k)|^2d\mathbf{x}\end{aligned}$$

(6)

It can be seen that the above defined local intensity fitting energy $E_{u_{k}}$ is minimized when the zero level set of $\phi$ matches the interface between the foreground $\varOmega _f$ and background $\varOmega _b$, as in the case shown in Fig. 1(b), and the constants $f_1(u_k)$ and $f_2(u_k)$ best approximate the image intensities I(x) in $\varOmega _{+}\cap \mathcal{O}_{u_k,r}$ and $\varOmega _{-}\cap \mathcal{O}_{u_k,r}$, respectively.

$$\begin{aligned} E(\phi ,f_1,f_2)=\sum _{k=1}^N E_{u_{k}}(\phi ,f_1(u_k),f_2(u_k)) \end{aligned}$$

(7)

In level set methods [8], a contour $\phi \subset \varOmega$ is represented by the zero level set of a Lipschitz function $\phi :\varOmega \rightarrow R$, which is called a level set function. In this paper, we let the level set function $\phi$ take positive and negative values outside and inside the contour $\mathcal{C}$, respectively. Let H be the Heaviside function, then the energy functional $E_{u_{k}}(\phi ,f_1(u_k),f_2(u_k))$ can be expressed as

$$\begin{aligned} E_{u_{k}}(\!\!&\phi ,f_1(u_k),f_2(u_k)) \nonumber \\&=\sum _{i=1}^{2}\lambda _i \int K_\sigma (\mathbf{x}-u_k)|I(\mathbf{x})-f_i(u_k)|^2M_i(\phi (\mathbf{x}))d\mathbf{x}\nonumber \\&+\nu \int |\nabla H(\phi (\mathbf{x}))|d\mathbf{x}\end{aligned}$$

(8)

where $M_1(\phi )=H(\phi )$ and $M_2(\phi )=1-H(\phi )$. Thus, the energy E in (7) can be written as following,

$$\begin{aligned} E(\phi ,f_1,f_2) = \sum \limits _{k=1}^N E_{u_{k}}(\phi ,f_1,f_2) \end{aligned}$$

(9)

thus,

$$\begin{aligned}&E(\phi ,f_1,f_2) \nonumber \\&=\sum _{i=1}^{2}\sum \limits _{k=1}^N \lambda _i (\int K_\sigma (\mathbf{x}-u_k)|I(\mathbf{x})-f_i(u_k)|^2 M_i(\phi (\mathbf{x})))d\mathbf{x}\nonumber \\&+\nu \int |\nabla H(\phi (\mathbf{x}))|d\mathbf{x}\end{aligned}$$

(10)

where the last term $\int |\nabla H(\phi ()\mathbf{x})|d\mathbf{x}$ computes the length of the zero level contour of $\phi$. The length of the zero level contour can be equivalently expressed as the integral $\int \delta (\phi )|\nabla \phi |d\mathbf{x}$ with the Dirac delta function $\phi$.

To preserve the regularity of the level set function $\phi$, which is necessary for accurate computation and stable level set evolution, we introduce a level set regularization term in our variation level set formulation. As proposed in Li’s article [4], we define the level set regularization term as

$$\begin{aligned} \mathcal{P}(\phi ) = \int \frac{1}{2}(|\nabla \phi (\mathbf{x})|-1)^2d\mathbf{x}\end{aligned}$$

(11)

which characterizes the deviation of the function $\phi$ from a signed distance function. Therefore, we propose to minimize the energy functional

$$\begin{aligned} \mathcal{F}(\phi , f_1,f_2)=E(\phi , f_1,f_2)+\mu \mathcal{P}(\phi ) \end{aligned}$$

(12)

where $\mu$ is a positive constant.

We minimize the energy functional $\mathcal{F}(\phi ,f_1, f_2)$ with respect to $\phi$ using the standard gradient descent method by solving the $gradient \; flow\; equation$ as follows:

$$\begin{aligned} \frac{\partial \phi }{\partial t}=&-\delta (\phi )(\lambda _1 e_1-\lambda _2 e_2) \nonumber +\nu \delta (\phi )\mathrm{div}(\frac{\nabla \phi }{|\nabla \phi |})\\ {}&+\mu (\nabla ^2\phi -\mathrm{div}(\frac{\nabla \phi }{|\nabla \phi |})) \end{aligned}$$

(13)

where $\delta$ is the Dirac delta function. In numerical implementation, the Dirac delta function is replaced by a smooth approximation as in [3], $e_1$ and $e_2$ are the functions

$$\begin{aligned} e_i(\mathbf{x})=\sum _{k=1}^N \int K_\sigma (\mathbf{x}-u_k)|I(\mathbf{x})-f_i(u_k)|^2d\mathbf{x}, \quad i = 1,2 \end{aligned}$$

(14)

where $f_1$ and $f_2$ are given by follows,

$$\begin{aligned} f_1(\mathbf{x})= \frac{\int K_\sigma (\mathbf{x}-u_k)I(\mathbf{x})H(\phi (\mathbf{x}))d\mathbf{x}}{\int K_\sigma (\mathbf{x}-u_k)H(\phi (\mathbf{x}))d\mathbf{x}} \end{aligned}$$

(15)

$$\begin{aligned} f_2(\mathbf{x})= \frac{\int K_\sigma (\mathbf{x}-u_k)I(\mathbf{x})(1-H(\phi (\mathbf{x})))d\mathbf{x}}{\int K_\sigma (\mathbf{x}-u_k)(1-H(\phi (\mathbf{x})))d\mathbf{x}} \end{aligned}$$

(16)

Fig. 1.

Two situations of the location of the zero level set (solid dark line) and the foreground (shaded region) and background (white region). (a) The zero level set does not match the interface between foreground and background. (b) The zero level set match the interface between foreground and background.

3 Experiment Results

Our method has been validated on the real vessel images and the data set for the Medical Image Computing and Computer Assisted Intervention (MICCAI) 2009 grand challenge on cardiac segmentation, respectively. Contour accuracies in terms of Dice Similarity Coefficient (DSC) are used to quantitatively evaluate the accuracy of our method. DSC is defined as

$$\begin{aligned} DSC = \frac{2 \times |R_1\cap R_2|}{|R_1|+|R_2|} \end{aligned}$$

(17)

where $|\cdot |$ is the area of a region, $R_1$ is the region obtained from a segmentation algorithm, and $R_2$ is a reference region. In the following experimental results, we use a segmented region of a segmentation algorithm as the reference region $R_2$ to compute its similarity with other segmentation results. It can be easily seen that a higher value of DSC indicates a better performance. The parameters in this paper are set as follows: $\sigma$ is set to 5.0 , $\epsilon$ is set to 1.0 , $\mu$ is set to 1.0. Others, $\lambda _1$, $\lambda _2$, $\nu$, and the time step $\triangle t$ are set to 1.0, 1.5, 0.004, 0.1, respectively.

The proposed IFOC algorithm has been tested on the number of images of different modalities. In this section, we show some examples mainly of medical images. For examples, Fig. 2 shows the segmentation results of IFOC on 8 test images: the real image of a T-shaped object suffers from non-uniform illumination; the synthetic image, two X-ray images of vessels are typical images with intensity inhomogeneity, and the second row are cardiac different modality images (including CT and MR). The final segmentation results in Fig. 2 show that the proposed IFOC method can achieve desirable results for all images with intensity inhomogeneities.

Fig. 2.

Performs of our new method for different images. The initial contours (green contours) and final segmentation results (red contours) are shown in every image. (Color figure online)

Fig. 3.

Robustness of our method to contour initializations is demonstrated by its results for a X-ray vessel image with different initial contours comparing with RSF model. Row 1: Segmentation result of RSF model; Row 2: Segmentation result of our new method; Row 3: Comparison of two methods in term of accuracy measured with DSC.(Color figure online)

Fig. 4.

Applications of our method to a cardiac MRI image at end-systole. Row 1: Segmentation result of RSF model; Row 2: Segmentation result of our new method; Row 3: Variation of the segmentation results from different contour initialization in terms of Dice similarity coefficient.

With the DSC metric, we can quantitatively evaluate the performance of our method with different initializations. We applied IFOC to X-ray vessel image (Fig. 3) and cardiac magnetic resonance (MR) image (Fig. 4) with 20 different initializations of the contour. Figures 3 and 4 demonstrate the robustness of proposed method IFOC to initialization. In Fig. 3, we show three of the 20 initial contours (green contours) and corresponding results (red contours). For comparison, we also applied RSF model in [3] to these images with the same initializations. It can be seen that despite the significant difference in the locations of initial contours, the proposed method precisely produces the same results, all accurately capturing the object boundaries, while the RSF model is sensitive to initialization. In this paper, we are not able to quantitatively evaluate the segmentation accuracy without ground truth obtained by manual segmentation. By comparing the IFOC with the RSF [3], we observed that the results of two methods are similar if the initial contours are placed appropriately. However, we found that the proposed IFOC algorithm is significantly less sensitive to contour initialization than RSF model. This can be quantitatively verified by computing the DSC between the segmentation results for 19 initializations with the 20th initialization as a reference. The boxplots in Figs. 3 and 4 show that the 19 DSC values of IFOC segmentation results are all very close to 1, while the 19 DSC values of RSF segmentation results significantly vary in a considerable large range. This experiment quantitatively demonstrate that IFOC is more robust to initialization.

4 Conclusions

In this paper, we introduce a novel region-based image segmentation method in a level set framework, which is not only effective and efficient when segment images with intensity inhomogeneity, but also has ideal performance in different initializations. The new algorithm IFOC reduce the dependence on initialization, significantly improve the robustness. Due to the sparse placement of the ISNs, our new algorithm is efficient than our previous work and eliminate an intrinsic drawback in our previous work. Experimental results have demonstrated the advantages of our new method over other methods in terms of accuracy, efficiency, and robustness.