Abstract
In this paper, we propose a new model for segmentation of both gray-scale and color images. This model is inspired by the GAC model, the region-scalable fitting model, the weighted bounded variation model and the active contour model based on the Mumford-Shah model. Compared with other active contour models, our new model cannot only make full use of advantages of both edge-based and region-based models, but also maintain more accurate overall message of segmented objects. Moreover, we establish the existence of the global minimum of the new energy functional and analyze the property of it. Finally, numerical results show the effectiveness of our proposed model.
Similar content being viewed by others
1 Introduction
The image segmentation problem is fundamental in the field of computer vision, and the aim of it is to divide an image into a finite number of important regions. Recently, variational methods have been extensively studied for image segmentation because of their flexibility in modeling and advantages in numerical implementation.
The geometric active contour (GAC) model is one of the most well-known variational models [1–4] for image segmentation. It was proposed in [2] and has been widely used in practice [5]. The main idea of the GAC model is to utilize the image gradient to stop evolving contours on object boundaries. However, this model has one major disadvantage, that is, giving an initial curve, during the evolution, the energy may evolve to its local minimizers.
Moreover, there are other variational models for image segmentation [6–8]. These models do not utilize the image gradient and are significantly less sensitive to the location of initial contours than other modes. Therefore, they have good performance for the image with weak object boundaries. Among these models, the Chan-Vese model [6] is very popular and useful. This model can be applied for the segmentation of images with two regions, each having a distinct mean of pixel intensities. In order to handle images with multiple regions, Vese and Chan proposed the piecewise constant (PC) models [9], in which multiple regions can be represented by multiple level set functions. And yet, these PC models are not very successful for images with intensity inhomogeneity. To deal with more general situation efficiently, Chunming Li proposed the region-scalable fitting model in [10]. However, this model is not always valid when the whole object needs to be segmented.
In this paper, inspired by the GAC model, the region-scalable fitting model, the weighted bounded variation model [3–5] and the active contour model based on the Mumford-Shah model [11], we propose a new model which can be applied to segment both gray-scale and color images. In order to make our model apply to color images, we shall use the red-green-blue model [12]. Although there are some other models which are similar to our new model, they have different essence and applications. Our new model cannot only make full use of advantages of both edge-based and region-based models, but also overcome the usual drawback in the level set approach. Moreover, compared with other active contours models, our model can keep more accurate overall message of segmented objects for simple and complex images of different modalities. Finally, we investigate the new model mathematically and establish the existence of the minimum to the new energy functional.
The remainder of the paper is organized as follows. In Section 2, we show some background. In Section 3, our new model is proposed. Theoretical results, iterating schemes and experimental results are also given in this section. Finally, we conclude our paper in Section 4.
2 Background
2.1 GAC model
Consider a given vector-valued image , where is the image domain and is the dimension of the vector . For gray-scale images, and for color images, . In [2, 13], the geodesic active contour model (GAC) is defined by the following minimization problem:
where ds is the Euclidean element of length and is the length of the curve c. Hence the energy functional in (2.1) is actually a new length obtained by weighting the Euclidean element of length ds. The function g is an edge indicator function that vanishes at object boundaries. If ,
where β is an arbitrary positive constant.
If , it is a color image and this stopping function should be modified. For a color image , a new stopping function is proposed as follows:
where ∧ is the largest eigenvalue of the structure tensor metric in the spatial-spectral space, and
where R, G and B represent the pixel values of red, green and blue after Gaussian convolution, respectively, i.e., , and .
2.2 The region-scalable fitting model
Let be the image domain, and let be a given gray-scale image. The region-scalable fitting model is defined by minimizing the following energy functional:
where is the Gaussian function and , are two functions that fit image intensities near the point x. Moreover, ϕ is the level set function embedding the evolving active contour and is the Heaviside function.
This model does not need to re-initialize ϕ periodically during the evolution because of the second term of (2.4). If , (2.4) is equivalent to
The steady state solution of this gradient flow is the same as
Moreover, it is known that equation (2.5) is the gradient descent flow of the following energy:
3 Our proposed model
Let be a given vector-valued image, where is the image domain and is the dimension of the vector . For gray-scale images, and for color images, . Inspired by the GAC model, energy functional (2.6) and the active contour model based on the Mumford-Shah model [11], our new model is constructed. This model is to minimize the following energy functional w.r.t. , :
where g is a diffusion coefficient defined in the same way as formula (2.2) or (2.3) and is the Gaussian function.
In the following, we analyze the above model from two aspects. Firstly, for any given u, according to the necessary condition of the minimization problem, the functions , must satisfy the following equations:
where takes larger values at the points near the center point y, and decreases to 0 as x goes away from y. Therefore, , are allowed to vary in space.
Furthermore, for any given , , model (3.1) can be converted into a simpler form. That is,
If u is limited to a characteristic function , energy functional (3.3) can be changed into the following form:
where C is a constant.
In this case, model (3.3) is equal to the following constrained minimization problem:
when approximating with spatially varying fitting functions , .
The above analysis shows that our new model uses not only the edge detector which contains information concerning the boundaries of objects, but also the spatially varying fitting functions , which are used to approximate the image intensities and avoid existence of the local minimizers to energy functional (3.3).
3.1 Mathematical results
In [11, 14], the proof of existence of models was not given. In the following, we state existence of the minimizer to energy functional (3.3) and analyze the property of it.
Theorem 1 For any given (, where ), there exists a function minimizing the energy functional E 1 in (3.3).
Proof Let
Since , we get . Since , , then we have
Assume and is the minimizing sequence of (3.3) in , i.e., . So, there is a positive constant such that
If , from (2.2) we can obtain with c being a positive constant. Since , where C 1 is a positive constant.
If , the structure tensor metric is symmetric positive and ∧ is the largest eigenvalue of . Thus
where (). Since (), , where C 2 is a positive constant.
In these two cases, we all obtain ( or ). Then
So, is bounded.
Since , the sequence has a bounded BV-norm.
Thus, there is a subsequence, also denoted by , and such that strongly in . Moreover, according to the formula in , we know that there is a subsequence, also denoted by , satisfying a.e. for . Since for any , a.e. for .
Assume , s.t. and , for any . Let
Then
That is, strongly in and for every .
Furthermore, by the lower semi-continuity for the space, we get
So, . According to the dominated convergence theorem, we know
These formulas (3.5)-(3.6) imply the weak lower semi-continuity of the energy functional E 1
Therefore and is a minimum of the energy functional E 1. □
Similar to [11, 15], we can obtain the property of minimizers to energy functional (3.3).
Theorem 2 Let (), . For any given , (), if is any minimum of the energy functional E 1 defined in (3.3), then for almost every , the characteristic function is a global minimum of the functional E 1 where c is the boundary of the set .
In addition, according to the above theorem, we know is a minimizer of the following minimization problem:
3.2 Numerical implementation
In the numerical algorithm, we do not deal with the new variation model (3.1) directly since too many equations need to be computed. In order to improve computational efficiency, we use the algorithm framework of the paper [14] to deal with our model. That is, we minimize the energy functional E by alternating the following steps:
-
(1)
Considering u fixed, compute and by using formula (3.2);
-
(2)
Considering and fixed, update u by using the iterative schemes of minimization problem (3.3).
When a steady state is found, the final segmentation is obtained by thresholding u at any level in (in our experiments, we choose ).
In the following, we give the iterative schemes of minimization problem (3.3). To solve this minimization problem, we firstly change it into the following unconstrained minimization problem:
where is an exact penalty function provided that the constant α is chosen large enough. The energy functional E 2 is convex, so E 2 does not possess local minimizers. Hence, any minimizer of E 2 is global.
Based on [16–20], we use a convex regularization as follows:
where θ is chosen to be small. Since this functional is convex, its minimum can be computed by minimizing this functional w.r.t. u and v separately. That is,
-
(1)
v being fixed, we search for u as a solution of
(3.8) -
(2)
u being fixed, we search for v as a solution of
(3.9)
According to [20], we know the solution of (3.8) can be given by
where is given by
The previous equation can be solved by the fixed point method
Moreover, the solution of (3.9) is given by
where
3.3 Experimental results
The proposed variational model can be applied to both gray-scale and color images. Firstly, we compare our new model with the GAC model in Figure 1. According to Figures 1c, d, we see that our proposed model can segment all the edges accurately. This result is hard to achieve by using the GAC model. Then we compare our proposed model with the model in [11] in Figures 2 and 3. Figures 2a and 3a are the original gray-scale images. The final active contours and u got by using our new model are displayed in Figures 2c, d and Figures 3c, d, respectively. Figures 2b and 3b show the active contours obtained by using the model in [11]. From these experiment results, we find that our model can segment the entire object more accurately and keep more details. Finally, we use our new model for segmentation of color images in Figures 4, 5, 6. Figure 4a is the noisy image with the variance 0.01. Figures 5a and 6a are complex images. According to Figures 4b, c, 5b, c, and 6b, c, we see that in all the three cases the experimental results are very good and can correspond to the actual needs.
4 Conclusion
This paper describes a new model for segmentation of gray-scale and color images. This model is based on the GAC model, the region-scalable fitting model, the weighted bounded variation model and the active contour model based on the Mumford-Shah model. Compared with other active contour models, our new model cannot only make full use of advantages of both edge-based and region-based models, but also keep more accurate overall message of the segmented objects. Our numerical results confirm the effectiveness of our algorithm. Moreover, we investigate the new model mathematically and establish the existence of the minimum to the new energy functional.
References
Kass M, Witkin A, Terzopoulos D: Snakes: active contour models. Int. J. Comput. Vis. 1988, 1: 321–331. 10.1007/BF00133570
Caselles V, Kimmel R, Sapiro G: Geodesic active contours. Int. J. Comput. Vis. 1997, 22: 61–79. 10.1023/A:1007979827043
Li C, Liu J, Fox MD: Segmentation of external force field for automatic initialization and splitting of snakes. Pattern Recognit. 2005, 11: 1947–1960.
Li C, Xu C, Gui C, Fox MD: Level set evolution without re-initialization: a new variational formulation. IEEE Conference on Computer Vision and Pattern Recognition 2005, 430–436.
Li F, Shen C, Pi L: A new diffusion-based variational model for image denoising and segmentation. J. Math. Imaging Vis. 2006, 26: 115–125. 10.1007/s10851-006-8303-2
Chan TF, Vese LA: Active contours without edges. IEEE Trans. Image Process. 2001, 38: 266–277.
Paragios N, Deriche R: Geodesic active regions and level set methods for supervised texture segmentation. Int. J. Comput. Vis. 2002, 46: 223–247. 10.1023/A:1014080923068
Tsai A, Yezzi A, Willsky AS: Curve evolution implementation of the Mumford-Shah functional for image segmentation denoising interpolation and magnification. IEEE Trans. Image Process. 2001, 10: 1169–1186. 10.1109/83.935033
Vese LA, Chan TF: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 2002, 50: 271–293. 10.1023/A:1020874308076
Li C, Kao C, Gore JC, Ding Z: Minimization of region-scalable fitting energy for image segmentation. IEEE Trans. Image Process. 2008, 10: 1940–1949.
Bresson X, Esedoglu S, Vandergheynst P, Thiran JP, Osher S: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 2007, 28: 151–167. 10.1007/s10851-007-0002-0
Blomgren PV, Chan TF: Color TV: total variation methods for restoration of vector valued images. IEEE Trans. Image Process. 1998, 7: 304–309. 10.1109/83.661180
Pi L, Shen C, Li F, Fan J: A variational formulation for segmenting desired objects in color images. Image Vis. Comput. 2007, 25: 1414–1421. 10.1016/j.imavis.2006.12.013
Mory B, Ardon R Lecture Notes in Computer Science. Fuzzy Region Competition: A Convex Two-Phase Segmentation Framework 2007, 214–226.
Chan TF, Esedoglu S, Nikolova M: Algorithms for finding global minimums of image segmentation and denoising models. SIAM J. Appl. Math. 2006, 66: 1632–1648. 10.1137/040615286
Chan TF, Golub GH, Mulet P: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 1999, 20(6):1964–1977. 10.1137/S1064827596299767
Carter, JL: Dual methods for total variation-based image restoration. PhD thesis, UCLA (2001)
Aujol JF, Chambolle A: Dual norms and image decomposition models. Int. J. Comput. Vis. 2005, 63: 85–104. 10.1007/s11263-005-4948-3
Aujol JF, Gilboa G, Chan TF, Osher S: Structure-texture image decomposition modeling algorithms and parameter selection. Int. J. Comput. Vis. 2006, 67: 111–136. 10.1007/s11263-006-4331-z
Chambolle A: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 2004, 20: 89–97.
Acknowledgements
This work is partially supported by the National Science Foundation of China (11271100, 11126222, 11301113, 71303067), the Fundamental Research Funds for the Central Universities (Grant No. HIT. NSRIF. 2012065, Grant No. HIT. HSS. 201201), China Postdoctoral Science Foundation funded project (Grant No. 2012M510933, Grant No. 2013M541400), the Heilongjiang Postdoctoral Fund (LBH-Z12102), the Aerospace Supported Fund of China (Contract No. 2011-HT-HGD-06), and also the Financial Support from Postdoctoral Science-Research Developmental Foundation of Heilongjiang Province (LBH-Q11111).
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
DZ, JS and YZ carried out the proof of the main part of this article, ZG corrected the manuscript, and participated in its design and coordination. All authors have read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Zhang, D., Sun, J., Guo, Z. et al. A new model for segmentation of gray-scale and color images. J Inequal Appl 2013, 556 (2013). https://doi.org/10.1186/1029-242X-2013-556
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1029-242X-2013-556