A variational level set model with closed-form solution for bimodal image segmentation

Abstract

In this work, we present a variational level set model with closed–form solution via combining with the fuzzy clustering method for robust and efficient image segmentation. For the designed energy functional, the two region parameters are first quickly pre–computed by means of the fuzzy c–means method and then embedded into a variational binary level set framework. Unlike the traditional variational level set models and optimization algorithms, our proposed model could directly obtain an exact closed–form solution of the level set function without using any iterative calculations and it is thus the globally optimal solution. Furthermore, we investigate the closed–form formula and achieve a significant property of the solution. As a byproduct, the manual initialization of the level set function and the sophisticated setting of time step in the process of numerical implementation are completely eliminated and thus leads to more robust segmentation results. Numerical experiments on both synthetic and real images verify the theoretical analysis of the proposed model and confirm the segmentation performance of the proposed method in terms of efficiency, accuracy and insensitiveness to parameters tuning.

1 Introduction

Image segmentation is one of the long–standing but challenging subjects in the communities of image processing and pattern recognition. The approaches of unsupervised image segmentation can automatically divide an image domain into coherent regions according to some computable visual attributes, such as the clustering method [1], superpixel/supervoxel methods [18, 32, 33], Gaussian mixture model (GMM) [7], graph theoretic methods [6, 9, 34], discrete high-order energy method [29, 35] and active contour/level set methods [12, 37,38,39, 47]. By clustering image pixels with their similar properties into distinct groups, image segmentation gives a concise profile of an image and extracts some compact and useful image features for boosting the subsequent middle- or high-level image processing tasks.

The general purpose of image segmentation is to yield a binary segmentation mask of the original image by extracting the foreground objects from its background. On the one hand, a vast number of literature on the work of graph-based approach have been proposed during the past few decades, of which includes superpixel/supervoxel methods [18, 32, 33], Gaussian mixture model (GMM) [7] and discrete high-order energy method [29, 35]. An given image is usually transformed into an undirected graph with their vertices denoting image pixels and the edges connecting pairs of vertices. Then, the image segmentation problem is equivalent to partitioning the vertices into disjoint segments. These graph based approaches can be formulated or solved by various algorithms, such as clustering [30], random walk [6, 9, 34], submodular function optimization method [30, 31]. The graph-based algorithms work well in aspect of adhering to boundaries but need to interactively scribble foreground and background pixels and suffer from low efficiency, which makes them inappropriate for automatic and real-time tasks. On the other hand, the variational level set (VLS) [4, 20,21,22] methods can label the boundaries of object regions with acceptable accuracy and thus attract significant attention. The core pith of VLS models is that they continuously delineate/locate object boundaries of the given image by evolving a curve. More specifically, these VLS methods drive an initial contour towards object boundaries under some constraints from encountered image (external energy) and the curve itself (internal energy). One distinctive significance of the level set approach [28] is that it has capability of automatically handling complex deformation of topological structures.

With the help of the level set method, Chan and Vese designed a well–known VLS model [4], named as CV model, which is a simplified variant of the classical Mumford–Shah model [27]. The CV model mainly utilizes the global region statistic information of the image to represent the data fitting energy and employs an alternative minimization (AM) method to compute the two region parameters (piecewise constants) and the level set function consecutively. Starting from the CV model, tremendous other improved or well-designed versions have been introduced [2, 3, 5, 8, 10, 11, 14,15,16,17, 26, 40, 41, 46]. Unfortunately, the CV model mainly suffers from the drawback of contour initialization sensitiveness and parameters sensitiveness due to its energy functional being non–convex, and thus different initializations or parameter settings will yield totally distinct segmentation results for the same image. Until now, many attempts to remedy the defect of initialization sensitiveness from the point of convex relaxation [2, 3, 5] or functional modifying [14, 17, 40] have been put into effect. For instance, inspired by the pioneering work [5], Bresson et al. [2] reported a convex relaxation method via introducing a fuzzy function and total variation (TV) regularization, which makes the proposed model attain global segmentation performance. However, the convergence rate of numerical scheme is very slow because of the utilization of TV regularization term. Thereafter, Brown et al. in [3] proposed an improved model and developed a fast algorithm by transforming the TV norm into its dual formulation. From the view of functional modifying, Lee and Seo [14] presented a modified version of the CV model (termed as LS model in this paper), which multiplies the level set function and the shifted Heaviside functions to control the movement of the level set function such that it avoids trapping into local minimum and restricts the value of level set function in a certain scope. Based on the LS model, Li and Kim [17] presented a novel VLS model (referred to as LK model in this paper) which utilizes a linear function as a substitute for the Heaviside function and designed an unconditionally stable semi–implicit numerical scheme to solve the proposed energy functional. Following the LK and LS models, we in [40] proposed a convex VLS model (referred to as GCoV model) by combining the coefficient of variation (CoV) and binary level set function. Although these VLS models outperform the CV model and are less sensitive to the contour initialization, the problem of initialization sensitiveness and parameter settings is remain unresolved completely.

As we all know, the CV model and all of the improved VLS models are typically solved via the AM scheme by firstly updating the two region parameters for keeping the level set function fixed and consecutively updating the level set function while keeping the two region parameters fixed. Thus, the AM method utilized to minimize the underlying energy functional may bring forth some essential drawbacks. Firstly, due to alternatively updating the unknown variables, a foremost problem is how to give the suitable iteration numbers for updating each class variables so that it achieves a trade–off between the segmentation efficiency and performance. Secondly, the AM algorithm may have very high computational overhead owing to alternatively computing the two class unknown variables. Thirdly, the evolution equation of level set function deduced from the steepest descent method usually needs to superpose an initial contour and to introduce a artificial time step, which indicates that the involving models may be sensitive to contour initialization and time step to some extent.

In an attempt to alleviate these limitations, we put forward a VLS model with closed–form solution for robustly and efficiently segmenting image in this work. The philosophy behind our method is that we first employ other efficient methods (such as fuzzy c–means method) to quickly pre–estimate the two prototypes of cluster centers of the entire image and then embed them into a variational binary level set framework. Different from the most existing models and algorithms, the proposed model could directly obtain the closed–form solution of the level set function without exploiting any iterative calculations and thus it could get the globally optimal solution. Furthermore, we investigate the closed–form formula and find a fascinating property of the solution. As a byproduct, the initialization of the level set function and the setting of time step are free of need any more and result in more robust segmentation performance. Experimental results on several synthetic and real images verify the theoretical analysis and confirm the segmentation performance of the proposed VLS method in terms of efficiency, accuracy and robustness against parameters tuning when comparing with the related VLS models.

The contributions and advantages of this study to variation level set model are summarized as follows:

• 1) Since the two region parameters are pre–computed, resulting the proposed energy functional reduced into an energy functional of signal variable, it is proved strictly convex without posing any additional constraints.

• 2) The minimization of the energy functional no longer needs to be solved by alternatively calculating the two region parameters and the level set function, the proposed method therefore arrives a fast computation speed for extraction of object boundaries.

• 3) The proposed energy functional could directly obtain the exact closed–form solution via variational calculus, which makes the proposed algorithm be free of setting initial contours and time step and thus finally achieving automatic segmentation results.

The remaining parts of this paper are structured as follows. In Section 2, we briefly review the related VLS models. Section 3 introduces the proposed VLS model and give a theoretical analysis. We in Section 3.2 solve the proposed model by utilizing the variational calculus and obtain the closed–form solution. Section 4 verifies the theoretical analysis of the proposed model and assesses the segmentation performance of the proposed method via numerical simulations on synthetic and real images. Finally, we discuss the proposed model and draw conclusion to this paper in Section 6.

2 Related VLS models

2.1 CV model

Chan and Vese in [4] proposed a famous VLS model which predigests the Mumford–Shah functional [27] via the level set function for the purpose of segmenting bimodal image. The formulation of the Chan–Vese model can be depicted as the following energy functional:

$$ \begin{array}{@{}rcl@{}} {E_{CV}}\left( {{c_{1}},{c_{2}},\phi } \right) &= &{\lambda_{1}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {c_{1}}} \right)}^{2}}H\left( {\phi \left( x \right)} \right)} dx \\ &&+ {\lambda_{2}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {c_{2}}} \right)}^{2}}\left( {1 - H\left( {\phi \left( x \right)} \right)} \right)} dx \\ &&+ \mu {\int}_{{\varOmega}} {\left| {\nabla H\left( {\phi \left( x \right)} \right)} \right|} dx, \end{array} $$

(1)

where λ₁, λ₂ and μ are three positive trade–off parameters, $I\left (x \right )$ denotes the input image to be segmented, ϕ is the level set function, and $H\left (\cdot \right )$ is the Heaviside function. Region parameters c₁ and c₂ represent the intensity mean values of the foreground and the background regions, respectively.

A commonly used method to minimize the energy functional (1) is to utilize the Euler–Lagrange equations and alternatively updates c₁, c₂ and ϕ with an artificial time variable as follows:

Keeping the level set function ϕ fixed and minimizing the energy functional (1) with respect to the constants c₁ and c₂ yields the following expressions for c₁ and c₂:

$$ {c_{1}}\left( \phi \right) = \frac{{{\int}_{{\varOmega}} {I\left( x \right){H}\left( {\phi \left( x \right)} \right)dx} }}{{{\int}_{{\varOmega}} {{H}\left( {\phi \left( x \right)} \right)dx} }}, {c_{2}}\left( \phi \right) = \frac{{{\int}_{{\varOmega}} {I\left( x \right)\left( {1 - {H}\left( {\phi \left( x \right)} \right)} \right)dx} }}{{{\int}_{{\varOmega}} {\left( {1 - {H}\left( {\phi \left( x \right)} \right)} \right)dx} }}. $$

(2)

Keeping the constants c₁ and c₂ fixed and minimizing the energy functional (1) with regard to the level set function ϕ leads to the following evolution equation of level set:

$$ \frac{{\partial \phi }}{{\partial t}} = \delta \left( \phi \right)\left[ {\nabla \left( {\frac{{\nabla \phi }}{{\left| {\nabla \phi } \right|}}} \right) - {\lambda_{1}}{{\left( {I\left( x \right) - {c_{1}}} \right)}^{2}} + {\lambda_{2}}{{\left( {I\left( x \right) - {c_{2}}} \right)}^{2}}} \right]. $$

(3)

The CV model works well in handling images with low–level noise and in locating object boundaries that can not depicted by gradients. However, the energy functional (1) is non–convex and thus often gets stuck into local minima. Besides, it needs to meticulously tune parameters λ₁, λ₂ and time step t for some complex images in order to achieve satisfactory results.

2.2 LS model

To overcome the non–convex shortcoming of the Chan–Vese model, Lee and Seo [14] creatively designed a novel energy functional via multiplying the level set function and two shifted Heaviside functions as follows:

$$ \begin{array}{@{}rcl@{}} {E_{LS}}\left( \phi \right) &=& {\lambda_{1}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {c_{1}}} \right)}^{2}}\phi \left( x \right)H\left( {\alpha + \phi \left( x \right)} \right)} dx \\ &&- {\lambda_{2}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {c_{2}}} \right)}^{2}}\phi \left( x \right)H\left( {\alpha - \phi \left( x \right)} \right)} dx, \end{array} $$

(4)

where λ₁ and λ₂ are two fixed positive weights, α is an arbitrarily given small positive constant. In the designed energy functional, the authors multiply the level set function ϕ to avoid computing a local minimum and shift the Heaviside function $H\left ({ \pm \phi } \right )$ by ∓ α to constrain the value of level set function ϕ in interval $\left [ { - \alpha ,\alpha } \right ]$. The minimization of the energy functional (4) leads to the following level set evolution equation is obtained as:

$$ \begin{array}{@{}rcl@{}} \frac{{\partial \phi }}{{\partial t}}& =& - {\lambda_{1}}{\left( {I\left( x \right) - {c_{1}}} \right)^{2}}\left( {H\left( {\alpha + \phi \left( x \right)} \right) + \phi \delta \left( {\alpha + \phi \left( x \right)} \right)} \right) \\ &&+ {\lambda_{2}}{\left( {I\left( x \right) - {c_{2}}} \right)^{2}}\left( {H\left( {\alpha - \phi \left( x \right)} \right) - \phi \delta \left( {\alpha - \phi \left( x \right)} \right)} \right). \end{array} $$

(5)

As with the CV model, the two constants c₁ and c₂ remain computed by (2).

The LS model performs well on bimodal images due to it could compute a stationary global minimum. However, it also needs to discreetly choose time step in the numerical solving process because of just simply employing the explicit finite difference method [17].

2.3 LK model

To overcome the defect of LS model, Li and Kim [17] hired a linear function to substitute the Heaviside function of the LS model and give the following energy functional:

$$ \begin{array}{@{}rcl@{}} {E_{LK}}\left( \phi \right) &=& {\lambda_{1}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {c_{1}}} \right)}^{2}}\phi \left( x \right){H_{c}}\left( {1 + \phi \left( x \right)} \right)} dx \\ &&- {\lambda_{2}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {c_{2}}} \right)}^{2}}\phi \left( x \right){H_{c}}\left( {1 - \phi \left( x \right)} \right)} dx, \end{array} $$

(6)

where λ₁ and λ₁ are two fixed positive parameters and the linear function ${H_{c}}\left (z \right )$ is defined by

$$ {H_{c}}\left( z \right) = \frac{{1 + z}}{2}. $$

(7)

By minimizing the energy functional (6), the following level set evolution equation is given by:

$$ \frac{{\partial \phi }}{{\partial t}} = - {\lambda_{1}}{\left( {I - {c_{1}}} \right)^{2}}\left( {1 + \phi } \right) + {\lambda_{2}}{\left( {I - {c_{2}}} \right)^{2}}\left( {1 - \phi } \right). $$

(8)

The two piecewise constants c₁ and c₂ are updated as

$$ {c_{1}}\left( \phi \right) = \frac{{{\int}_{{\varOmega}} {I\left( x \right){H_{c}}\left( {\phi \left( x \right)} \right)dx} }}{{{\int}_{{\varOmega}} {{H_{c}}\left( {\phi \left( x \right)} \right)dx} }}, {c_{2}}\left( \phi \right) = \frac{{{\int}_{{\varOmega}} {I\left( x \right)\left( {1 - {H_{c}}\left( {\phi \left( x \right)} \right)} \right)dx} }}{{{\int}_{{\varOmega}} {\left( {1 - {H_{c}}\left( {\phi \left( x \right)} \right)} \right)dx} }}. $$

(9)

The LK model can better deal with the segmentation of bimodal image and is also insensitive to the time step due to it is solved via an unconditional stable semi–implicit numerical algorithm.

2.4 GCoV model

In [40], we present a novel VLS model motivated by the aforementioned models via integrating the CoV and the binary level function into the variational framework as follows:

$$ \begin{array}{@{}rcl@{}} {E_{WH}}\left( \phi \right) &= & \lambda {\int}_{{\varOmega}} {\frac{{{{\left( {I\left( x \right) - {c_{1}}} \right)}^{2}}}}{{{c_{1}^{2}}}}} {\left( {\phi \left( x \right) + 1} \right)^{2}}dx \\ & &+ {\int}_{{\varOmega}} {\frac{{{{\left( {I\left( x \right) - {c_{2}}} \right)}^{2}}}}{{{c_{2}^{2}}}}} {\left( {\phi \left( x \right) - 1} \right)^{2}}dx, \end{array} $$

(10)

where λ is a positive weight, and c₁, c₂ are two region parameters.

The minimization of the above energy functional with regard to ϕ results in the following evolution equation of level set function:

$$ \frac{{\partial \phi }}{{\partial t}} = - \lambda \frac{{{{\left( {I - {c_{1}}} \right)}^{2}}}}{{{c_{1}^{2}}}}\left( {\phi + 1} \right) - \frac{{{{\left( {I - {c_{2}}} \right)}^{2}}}}{{{c_{2}^{2}}}}\left( {\phi - 1} \right). $$

(11)

The two region constants c₁ and c₂ are updated by

$$ {c_{1}}\left( \phi \right) = \frac{{{\int}_{{\varOmega}} {I{{(x)}^{2}} \cdot H\left( {\phi (x)} \right)dx} }}{{{\int}_{{\varOmega}} {I(x) \cdot H\left( {\phi (x)} \right)dx} }}, {c_{2}}\left( \phi \right) = \frac{{{\int}_{{\varOmega}} {I{{(x)}^{2}} \cdot \left( {1 - H\left( {\phi (x)} \right)} \right)dx} }}{{{\int}_{{\varOmega}} {I(x) \cdot \left( {1 - H\left( {\phi (x)} \right)} \right)dx} }}. $$

(12)

The GCoV model achieves good segmentation performance and has a fast convergence speed because the large time steps can be chosen in the proposed numerical scheme. However, the proposed algorithm remain need to finely tune the parameter λ for some complex images and the time step can also not been set arbitrarily large.

It should be noted that the above mentioned four models and most classical VLS models usually alternatively update the two–class unknown variables, that is, they firstly update the two piecewise constants while keeping the level set function fixed and subsequently update the level set function for the two constants keeping fixed and until the zero level set arrive at the object boundaries. Therefore, the involving algorithms inevitably encounter the core problem of how to set appropriate iteration numbers for updating two–class unknown variables. Additionally, the evolution equation of the level set derived from gradient descent flow needs to set time step that is also a sophisticated technique.

3 Proposed method

3.1 Proposed model

In this section, we present a VLS model in which the prototype of two cluster centers is pre–computed via the fuzzy c–means (FCM) method, and thus the proposed model turns out to be an energy functional of single variable regarding the level set function.

Define Ω ⊂ R² as a bounded connected open region with Lipschitz continuous boundary, I : Ω → R and ϕ : Ω → R denote the original image and the level set function, respectively. The proposed energy functional can be formulated as follows:

$$ \begin{array}{@{}rcl@{}} E\left( \phi \right) &= & {\lambda_{1}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {m_{1}}} \right)}^{2}}{{\left( {\phi \left( x \right) + 1} \right)}^{2}}dx} \\ &&+ {\lambda_{2}}{\int}_{{\varOmega}} {{{\left( {I\left( x \right) - {m_{2}}} \right)}^{2}}{{\left( {\phi \left( x \right) - 1} \right)}^{2}}dx}, \end{array} $$

(13)

where λ₁, λ₂ are positive weighting parameters, m₁, m₂ are two prototypes of cluster centers to be computed and $\phi \in {L^{2}}\left ({\varOmega } \right )$. As for the proposed energy (13), the two prototypes of cluster center could be previously estimated via other clustering methods such as k–means [25], fuzzy c–means [1] and self–organizing maps (SOM) [36]. In this work, we utilize the fuzzy c–means to compute the prototype value of two cluster centers owing to its accuracy and convenience. Solving the energy functional (13) with respect to the level set function ϕ would gives birth to a optimal solution ϕ^∗ whose zero level set comes to be the contour line that separates the object regions from background.

Since the proposed energy functional only involves the level set function, we present the theoretical analysis (convexity) of the variational problem (13).

Theorem 1

Suppose $I \in {L^{2}}\left ({\varOmega } \right )$, the presented energy functional $E\left (\phi \right )$ is strictly convex.

Proof

See Appendix A. □

The proposed energy functional in this work is similar but has a little difference with our previous work [40] in details. We note that the two piecewise constants defined in our previously proposed model are depended on the level set function, which iteratively updates by following each evolution of the level set function. Thus, our previously proposed model [40] could just be proved convex under the constraint that the two region parameters are fixed. However, since the two region parameters in our proposed energy functional are pre–obtained by the fuzzy c–means method, which makes the two piecewise constants be independent of the level set function, the proposed energy functional is proved strictly convex.

3.2 Closed-form solution

We from Theorem 1 in Section 3 know that our designed energy functional $E\left (\phi \right )$ has a unique global optimum for any bimodal image $I \in {L^{2}}\left ({\varOmega } \right )$. To obtain the global optimum, we deduce the Gâteaux derivative of the energy functional $E\left (\phi \right )$ by using the theory of variational calculus.

The Gâteaux derivative of $E\left (\phi \right )$ at ϕ along the direction of a test function $\varphi \in C_{0}^{\infty } \left ({\varOmega } \right )$ can be expressed as

$$ \begin{array}{@{}rcl@{}} {E^{\prime}}\left( \phi \right)\varphi &=& \mathop {\lim }\limits_{\varepsilon \to 0} \frac{{E\left( {\phi + \varepsilon \varphi} \right) - E\left( \phi \right)}}{\varepsilon } \\ &=& {\left\langle {{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}}\left( {\phi + 1} \right) + {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}\left( {\phi - 1} \right),\varphi} \right\rangle_{{L^{2}}\left( {\varOmega} \right)}}, \end{array} $$

(14)

where ${\left \langle { \cdot , \cdot } \right \rangle _{{L^{2}}\left ({\varOmega } \right )}}$ denotes the inner product of ${L^{2}}\left ({\varOmega } \right )$. Based on the Riesz representation theorem, the Gâteaux derivative ${E^{\prime }}\left (\phi \right )\varphi$ can be deemed as the unique element representing the bounded linear functional ${E^{\prime }}\left (\phi \right )$ in ${L^{2}}\left ({\varOmega } \right )$ as follows:

$$ {E^{\prime}}\left( \phi \right)\varphi = {\left\langle {\nabla E\left( \phi \right),\varphi} \right\rangle_{{L^{2}}\left( {\varOmega} \right)}}, $$

(15)

where $\nabla E\left (\phi \right )$ means the gradient of $E\left (\phi \right )$ in ${L^{2}}\left ({\varOmega } \right )$. We could obtain the formulation of $\nabla E\left (\phi \right )$ by comparing (14) and (15) as follows:

$$ \nabla E\left( \phi \right) = {\lambda_{1}}{\left( {I - {m_{1}}} \right)^{2}}\left( {\phi + 1} \right) + {\lambda_{2}}{\left( {I - {m_{2}}} \right)^{2}}\left( {\phi - 1} \right). $$

(16)

In Theorem 2 below, we will prove that the proposed energy functional can directly obtain the closed-form solution and thus it is globally optima.

Theorem 2

The optimization of energy functional $E\left (\phi \right )$ in (13) has the closed-form solution

$$ {\phi^{*}} = - \frac{{{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}} - {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}}}{{{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}} + {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}}}, $$

(17)

and it holds

$$ \left| {{\phi^{*}}\left( x \right)} \right| \le 1, {\textbf{x}} \in {\varOmega}. $$

(18)

Proof

See Appendix A. □

3.3 About the regularization

For the proposed model, we do not pose any regularization to the level set function, hence the proposed model may not handle well complex images with non–regular object edges or outliers. To make up this deficiency, a natural method is that we can utilize a Gaussian filtering [45] to further smooth the level set function as the closed–form solution of level set function obtained. The formulation of Gaussian regularized level set is described as follow:

$$ {\phi_{s}^{*}} = {G_{\sigma} } * {\phi^ * }, $$

(19)

where ϕ^∗ is the level set function obtained form (23), G_σ denotes the Gaussian kernel with scale σ. By this way, the Gaussian filtering regularization can prevent the oscillation of level set when encountering non–regular object edges or outliers.

In Fig. 1, we provide a test experiment to verified the performance of the proposed model with or without Gaussian filtering process when dealing with complex images. The first column of Fig. 1 shows four images which has non-regular object boundaries or suffer from outliers. The segmentation results obtained by the proposed model without and with regularization are shown in the second column and third column, respectively. We can clearly see that the segmentation results obtained by the proposed model without filtering process are unsatisfactory. Conversely, the proposed model with additional filtering process extracts the objects acceptably. Therefore, it is remarked that we can further take the smoothness of the level set function as an additional step of the proposed algorithm in order to get better segmentation results.

Fig. 1

Segmentation results by the proposed model without and with using regularization for four images with non–regular object boundaries or outliers. First column: original images; Second column: segmentation results of the propose model without using any regularization; Last column: segmentation results of the proposed model with smooth regularization

We finally list the algorithm steps in detail to conclude the proposed method as Algorithm 1.

4 Numerical experiments

This section implements the proposed variational segmentation model by using the MATLAB 2018b software in a personal computer with intel(R) Core(TM) i7-7700, CPU 3.60GHZ and memory 16.00GB and performs a series of different numerical experiments on both synthetic and real images for the aim of verifying the theoretical analysis of the proposed method and assessing the segmentation performance when comparing with the related models such as CV model [4], LS model [14], LK model [17] and GCoV model [40]. Since the proposed model is insensitive to the setting of two parameters λ₁, λ₂ which would be justified in Fig. 8, we fix λ₁ = λ₂ = 1 in our method for all our experiments.

4.1 Performances of proposed method

In the following, different types of numerical experiment are designed to demonstrate the performance of the proposed model and algorithm.

Theorem 1-2 in Sections 3 and 3.2 told us that the proposed model could get an exact closed-form solution and the value of which is restricted in the range [− 1,1] for any bimodal image. To verify these theoretical results of the proposed method, we illustrate in Fig. 2 two simple experimental results with our proposed model segmenting an ideal binary image and a real bimodal image. The second column and the third column of Fig. 2 show the segmentation results and 3D–plots of level set function for the proposed model, respectively. We observe from the last column of Fig. 2 that the value of the level set function is indeed within the range [− 1,1] for the tested images, which is consistent with the theoretical analysis.

Fig. 2

Segmentation results of the proposed model for two bimodal images. First column: original images; Second column: segmentation results; Last column: 3D–plots of final level set function

To assess the performance of our proposed method in handling images without or with noise, we demonstrate the segmentation results of three images in Fig. 3. The first column of Fig. 3 shows two synthetic images with Guassian noise (zero mean and variance 0.005 and 0.01, respectively) and a real plane image. The second and third columns of Fig. 3 display the corresponding segmentation results the proposed method and 3D-plots of final level set functions, respectively. As can be observed, the proposed model works well on these images as the noise level is not high.

Fig. 3

Segmentation results by our method for images without or with noise. First column: original images. Second column: corresponding segmentation results; Last column: 3D–plots of final level set function

As the last numerical example in this subsection, the proposed model is applied to tackle images with blurry boundaries. In Fig. 4, we show three original images with weak edges in the first column and the final segmentation results of our proposed model in the second column, respectively. It can be clearly observed that the proposed method achieves satisfactory segmentation performance for these tested images.

Fig. 4

Segmentation results by our method for three real images with blurry boundaries. First column: original images. Second column: corresponding segmentation results; Last column: 3D–plots of final level set function

4.2 Comparison with FCM

The proposed model is very related with the FCM method due to the two region parameters are computed by FCM. To compare the segmentation performance of our proposed method and the FCM method, we in Fig. 5 demonstrate the segmentation results of two competing methods for four real images. The top row of Fig. 5 shows the original image, the middle and bottom rows illustrate the corresponding segmentation results of the FCM method and our proposed method, respectively. We can observe from the middle row that the FCM algorithm obtain unsatisfactory segmentation results, in which either existing non-smooth object boundaries or appearing some noise points around the objects. On the contrary, our proposed method can extract the desired objects acceptably. In addition, we list the CPU runing times of the FCM and the proposed model in Table 1. We can observe that the CPU times of the FCM is longer than that of our proposed model.

Fig. 5

Segmentation results by the FCM method and our proposed method for four real images. Top row: original images; Middle row: segmentation results of the FCM algorithm; Bottom row: our proposed method

Table 1 CPU times (in seconds) of FCM and our algorithm for four images in Fig. 5

4.3 Comparisons with related VLS models

In this subsection, real images selected from different modalities are employed to compare the proposed model with other related VLS models in terms of the efficiency, the accuracy and the robustness against parameters.

In practice, the computation efficiency is important for image segmentation system, which is usually determined by the segmentation scheme. We give detailed analysis of computation complexity for our proposed algorithm and compare with other related VLS algorithms [4, 14, 17, 40]. Assume a gray image with size of M × N. Our proposed algorithm consists of two stages involving the computation of two region parameters and the final level set closed-form solution by the FCM and (23), respectively. For the FCM algorithm, the computation complexity is O(MNkT) [1], where k is the number of cluster center, and T denotes the iteration number. For the computation of closed-form solution, the computation complexity is O(MN) according to the (23). Thus, the total computation complexity of our proposed algorithm is O(MNkT). Table 2 lists the compared results of computation cost with other related VLS algorithms [4, 14, 17, 40]. In Table 2, T₁, T₂, T₃ and T₄ represent different iteration numbers for corresponding segmentation algorithms, and they are determined by the convergence speed of their corresponding segmentation algorithms.

Table 2 Complexity analysis of different segmentation algorithms

To evaluate the efficiency comparison of the proposed model and other four contrastive models, we illustrate the segmentation results by four compared models and our model for three plane images and four other real images in Fig. 6, and we observe that all of models obtain the satisfied segmentation results. Furthermore, we in Table 3 display the corresponding computational CPU times (in seconds) for each of the five compared models. The proposed model is faster than the CV model and the LS model, but slower than the LK model and the WH model. However, we observe that the high computational overhead is mainly concentrates on estimating two values of cluster centers via the fuzzy c–means method (Stage 1), which is shown in Table 4, and the final closed-form solution of level set function (Stage 2) has low computation consuming. A good trick to alleviate this limitation is that we can pre–obtain the clustering centers for all of images to be segmented through parallel computing and then solve the energy functional of level set function. By this way, the proposed method is applicable to modularize and further easily integrated into the automatic segmentation system.

Fig. 6

Segmentation results by five competing models for seven real images. First column: original images; From the second column to the last column: segmentation results of CV model, LS model, LK model, GCoV model and ours model, respectively

Table 3 CPU times (in seconds) of five competing algorithms for seven images in Fig. 6

Table 4 CPU times (in seconds) of two stages in our algorithm for seven images in Fig. 6

In Fig. 7, we compare the segmentation accuracy of our model with other four competing models. Three real images are shown in the first column of Fig. 7, and the corresponding segmentation results of the CV model, the LS model, the LK model, the WH model and our proposed model are demonstrated in the second column to the last column, respectively. We can see from the second column to the fourth column that the CV model, the LS model and the LK model perform weak and give uncorrect segmentation results. The segmentation results of the WH model shown in the fifth column are visually better than the ones of other three models, but there still exist some inaccurate edges delineation. However, it is clearly illustrated that our proposed model yields the best segmentation results than other four models.

Fig. 7

Segmentation results by five competing models for three real images. First column: original images; From the second column to the last column: segmentation results of CV model, LS model, LK model, GCoV model and ours proposed model, respectively

Next, we test the parameters robustness of the proposed model and other four compared models. In Fig. 8, we present the segmentation performance for a real image with two boats by five competing models with setting four pair different parameters λ₁ and λ₂. In this experiment, we choose $\left ({{\lambda _{1}},{\lambda _{2}}} \right ) = \left ({1,2} \right )$, $\left ({{\lambda _{1}},{\lambda _{2}}} \right ) = \left ({2,1} \right )$, $\left ({{\lambda _{1}},{\lambda _{2}}} \right ) = \left ({2,3} \right )$ and $\left ({{\lambda _{1}},{\lambda _{2}}} \right ) = \left ({3,2} \right )$, respectively, for each row of Fig. 8, and we demonstrate the segmentation results and corresponding final 3D-plot of level set function obtained by five models from the left column to the right column, respectively. It is clearly observed that the four comparing models obtain different segmentation performance for different parameters setting, which indicates that these methods are sensitive to the selection of parameters λ₁ and λ₂, that is, one need to finely tune the parameters in order to yield the best results, and thus these methods are inapplicable to automatic segmentation process. Conversely, we from the last column can conclude that our proposed method gets the visually identical segmentation results for different parameters.

Fig. 8

Robustness comparison to parameters λ₁ and λ₂ for five models. First column: original images. Second column to Last column: segmentation results and corresponding 3D–plots of level set function of CV model, LS model, LK model, GCov model and ours method, respectively, for different parameters choosing

Since the numerical scheme presented for the LK model and WH model is proved to be unconditionally stable, we just need to compare the insensitiveness of the proposed model and other two models to the time step. Figure 9 displays the segmentation results of a real image by three competing models with choosing five different time steps t = 0.01,0.05,0.1,0.5,1, respectively, from the left column to the right column for every row. The second row and third row depict the segmentation results and the corresponding 3D-plots of level set function yielded by the CV model. The fourth row and fifth row give the segmentation results and the corresponding 3D-plots of level set function obtained by the LS model. As can be seen, these two models obtain inconsistent segmentation performance for different parameters setting. Nevertheless, we from the last row can state that our model achieves better segmentation performance due to it is completely free of setting time step.

Fig. 9

Robustness comparison to time step t for CV model, LS model and our model. First row: test images; Second row and Third row: segmentation results and corresponding 3D–plots of level set function by CV model with different time step; Fourth row and Fifth row: segmentation results and corresponding 3D–plots of level set function by LS model with different time step; Last row: segmentation result and corresponding 3–plot of level set function by our method

5 Discussion

It is worthy mentioning that our proposed method is only built on the simple intensity features computed from images by the FCM method. Actually, complex images in the real-world such as inhomogeneous images and natural images need more high-level features to represent, the proposed model thus cannot handle such images well in its current form. Recently, Convolutional Neural Networks (CNNs) as a type of powerful visual models of learning features from data has witnessed a new trend of development [13, 19, 23, 24, 42]. CNNs emphasizes the importance of automatic hierarchical feature extraction and end-to-end task learning, which can yield more high-level features by assembling low-level features layer by layer. Provided with enough training data, deep learning architectures have been shown impressive performance on a diverse set of visual tasks, involving the global scale image classification [13], semantic segmentation [23, 24, 44], video tracking [19] and salience prediction [43]. In the future work, we would investigate more the integration of the level set method with other machine learning methods or deep learning technology to obtain the salient features of image and design high-performance model so that handle well the images under more complex situation.

6 Conclusions

By combining the advantage of fuzzy c–means method, we in this paper propose a very robust and efficient VLS model for segmenting the bimodal image. Unlike the conventional VLS methods utilizing alternating minimization scheme, we first harness fuzzy c–means algorithm to obtain an approximate cartoon structure of the encountered image and from which we can previously estimate the intensity average values of foreground and background regions. Then the two obtained prototype values are integrated into the variational binary level set framework, which leads to an energy functional of single variable with regard to the level set function. By this way, the proposed model could be proved strictly convex and produce a global segmentation result with a closed–form solution. The numerical simulations perform on both synthetic and natural images confirm the computation effectiveness, segmentation accurateness and parameters robustness of the proposed method, yielding excellent segmentation performance when comparing with other related VLS models.

Appendix A

Proof of Theorem 1

Proof

∀m,n > 0 with m + n = 1. For given ${\phi _{1}}, {\phi _{2}} \in {L^{2}}\left ({\varOmega } \right )$ and ϕ₁≠ϕ₂, we have

$$ \begin{array}{@{}rcl@{}} {\left( {m{\phi_{1}} + n{\phi_{2}} + 1} \right)^{2}} &=& {\left( {m\left( {{\phi_{1}} + 1} \right) + n\left( {{\phi_{2}} + 1} \right)} \right)^{2}} \\ &=& {m^{2}}{\left( {{\phi_{1}} + 1} \right)^{2}} + {n^{2}}{\left( {{\phi_{2}} + 1} \right)^{2}} + 2mn\left( {{\phi_{1}} + 1} \right)\left( {{\phi_{2}} + 1} \right) \\ &<& {m^{2}}{\left( {{\phi_{1}} + 1} \right)^{2}} + {n^{2}}{\left( {{\phi_{2}} + 1} \right)^{2}} + mn\left( {{{\left( {{\phi_{1}} + 1} \right)}^{2}} + {{\left( {{\phi_{2}} + 1} \right)}^{2}}} \right) \\ &=& m\left( {m + n} \right){\left( {{\phi_{1}} + 1} \right)^{2}} + n\left( {n + m} \right){\left( {{\phi_{2}} + 1} \right)^{2}} \\ &=& m{\left( {{\phi_{1}} + 1} \right)^{2}} + n{\left( {{\phi_{2}} + 1} \right)^{2}}. \end{array} $$

(20)

Analogous with (20), we obtain

$$ \begin{array}{@{}rcl@{}} {\left( {m{\phi_{1}} + n{\phi_{2}} - 1} \right)^{2}}& =& {\left( {m\left( {{\phi_{1}} - 1} \right) + n\left( {{\phi_{2}} - 1} \right)} \right)^{2}} \\ &<& m{\left( {{\phi_{1}} - 1} \right)^{2}} + n{\left( {{\phi_{2}} - 1} \right)^{2}}. \end{array} $$

(21)

combining (20) and (21), we get the relation

$$ E\left( {m{\phi_{1}} + n{\phi_{2}}} \right) < mE\left( {{\phi_{1}}} \right) + nE\left( {{\phi_{2}}} \right). $$

Hence, $E\left (\phi \right )$ is strictly convex in ${L^{2}}\left ({\varOmega } \right )$. The proof is completed. □

Proof of Theorem 2

Proof

According to Theorem 1, we know that the proposed energy functional (13) has unique global minimum ϕ^∗. Therefore, we from the expression (16) can directly derive the following relation:

$$ {\lambda_{1}}{\left( {I - {m_{1}}} \right)^{2}}\left( {{\phi^{*}} + 1} \right) + {\lambda_{2}}{\left( {I - {m_{2}}} \right)^{2}}\left( {{\phi^{*}} - 1} \right) = 0. $$

(22)

Then we get the closed–form solution by explicitly solving the (22) as follows:

$$ {\phi^{*}} = - \frac{{{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}} - {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}}}{{{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}} + {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}}}. $$

(23)

At the meantime, we can obtain the following relation

$$ \begin{array}{@{}rcl@{}} \left| {{\phi^{*}}} \right| &=& \left| { - \frac{{{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}} - {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}}}{{{\lambda_{1}}{{\left( {I - {m_{1}}} \right)}^{2}} + {\lambda_{2}}{{\left( {I - {m_{2}}} \right)}^{2}}}}} \right| \\ &\le & \frac{{{\lambda_{1}}\left| {{{\left( {I - {m_{1}}} \right)}^{2}}} \right| + {\lambda_{2}}\left| {{{\left( {I - {m_{2}}} \right)}^{2}}} \right|}}{{{\lambda_{1}}\left| {{{\left( {I - {m_{1}}} \right)}^{2}}} \right| + {\lambda_{2}}\left| {{{\left( {I - {m_{2}}} \right)}^{2}}} \right|}} \\ &= & 1. \end{array} $$

(24)

This completes the proof. □