Abstract
Image segmentation is the initial step for every image analysis task. A large variety of segmentation algorithm has been proposed in the literature during several decades with some mixed success. Among them, the fuzzy energy-based active contour models get attention to the researchers during last decade which results in development of various methods. A good segmentation algorithm should perform well in a large number of images containing noise, blur, low contrast, region in-homogeneity, etc. However, the performances of the most of the existing fuzzy energy-based active contour models have been evaluated typically on the limited number of images. In this article, our aim is to review the existing fuzzy active contour models from the theoretical point of view and also evaluate them experimentally on a large set of images under the various conditions. The analysis under a large variety of images provides objective insight into the strengths and weaknesses of various fuzzy active contour models. Finally, we discuss several issues and future research direction on this particular topic.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Image segmentation is a fundamental task in image analysis, computer vision, medical image processing, etc (Gonzalez and Woods 2008; Garcia-Lamont et al. 2018). Segmentation is a process of partitioning an image into various regions which are homogeneous with respect to their features (e.g., intensity, color, texture, etc) (Gonzalez and Woods 2008; Zaitoun and Aqel 2015). Various image segmentation algorithms have been developed during several decades (Fu and Mui 1981; Sahoo et al. 1988; Pal and Pal 1993; Khan 2014; Zaitoun and Aqel 2015). Among them, clustering and active contour models (acms) are most commonly used for image segmentation. Fuzzy logic has been used to solve various problems: decision making (Amin et al. 2019; Fahmi et al. 2019), pattern recognition (Melin 2018; Mitchell 2005), image segmentation (Naz et al. 2010; Zhang et al. 2017), etc. Fuzzy clustering has been successfully considered from the early stage of the image segmentation task up to now (Dunn 1973; Bezdek 1981). It can retain more information from the original image than crisp clustering by introducing the degree of belongingness of each image pixel to the clusters (Bezdek 1981). Fuzzy clustering using global image information is not robust for images which are corrupted by various types of noise (Dunn 1973; Bezdek 1981). A large number of modified fuzzy clustering techniques have been proposed by incorporating local information which is derived from the image to improve segmentation accuracy (Cai et al. 2007; Krinidis and Chatzis 2010; Gong et al. 2013; Liu et al. 2015; Zhang et al. 2017). Due to incorporation of local information, these methods are robust to the noisy environment to some extend.
Active contour model (acm) developed by Kass et al. (1988) is successfully applied for image segmentation task (Nguyen et al. 2012). The main idea of this technique is deformation of initial curve. Finally, it evolves toward the object boundary under some constraints. It produces closed parametric curve which represents object boundary (Cremers et al. 2007). Results are highly depend on the initial contour position and model is not robust for noisy and/or blurred images. To overcome these problems, various modified acms are invented in the literature (Caselles et al. 1993; Malladi et al. 1995; Caselles et al. 1997; Yezzi et al. 1997; Cohen and Kimmel 1997; Li et al. 2010; Gunn and Nixon 1997; Zhang et al. 2013). acms consider image gradient to attract the contour toward object boundary. They fail to extract the contour of noisy, blurred and discontinuous edged images.
Different from image gradient-based acm models, Chan and Vese (2001) proposed an active contour (referred as, Chen-Vese) model which depends on the region information of the image. In this model, two regions inside and outside of the contour are assumed to be homogeneous. They formulated it as an energy minimization problem and included global image statistics in this energy function. The energy function is then transferred into level set formulation. Chen-Vese model is robust to the initial contour position and noisy, blurred and discontinuous edged images. However, Chen-Vese model fails to segment images having intensity in-homogeneity. To solve this problem, various modification on Chen-Vese model have been done by incorporating local image statistics (Li et al. 2008; Lankton and Tannenbaum 2008; Zhang et al. 2010a, b; Wang et al. 2017).
Krinidis and Chatzis (2009) formulated region-based active contour model as a minimization of fuzzy energy function different from Chen-Vese formulation. This model is referred to as fuzzy energy-based active contour model which can handle objects whose boundaries are not necessarily defined by gradient, objects with very smooth or even with discontinuous boundaries. The fuzzy logic has been intensively used in clustering (e.g., image segmentation) but not in active contour models. Generally, fuzzy methods provide more accurate and robust clustering, and thus the authors combine fuzzy logic with active contour method to introduce fuzzy energy-based active contour model to segment images. The fuzziness of this energy function has the ability to reject local minima, and it is able to produce better segmentation results. Recent days, the researchers developed various new fuzzy energy-based models (Shyu et al. 2012a; Tran et al. 2014a, b; Mondal et al. 2016a) by modifying the base model proposed by Krinidis and Chatzis (2009) which are able to segment images in various complex environments. Recently, the fuzzy energy-based active contour models get attention from the researchers for image segmentation tasks. Image segmentation using fuzzy energy-based active contour models has indeed progressed to impressive, even amazing individual results. But as long as most of image segmentation papers (using fuzzy energy-based active contour models) still use a limited number of images to test the performance of their approach, it is very difficult to conclude anything on the robustness of the methods in a variety of circumstances. We feel that the time is ripe for an experimental survey for various conditions.
The aim of this survey is to access the state-of-the-art fuzzy energy-based active models for image segmentation with an emphasis on the accuracy and the robustness of the models on varying environments. We aim to group the models on the basis of global or local or both image information considered to formulate energy function. We also experimentally evaluate the performance of the existing models on the large number of images having various complexities with respect to various measures: Jacard error and F-measure. We have gathered 100 images of various categories under varying environments. Finally, we provide several issues associated these methods and future research direction. We hope that this survey will help the reader to understand insight into the strengths and weaknesses of various fuzzy energy-based active contour models from both the theoretical and practical point of views.
Rest of the paper is organized as follows. We discuss global fuzzy energy-based active models where energy term is defined with the global image feature in Sect. 2. Section 3 reviews on various local fuzzy energy-based active contour models while Sect. 4 analyzes global and local fuzzy energy-based active models from theoretical point of view. Experimental results and performance analysis of the existing fuzzy active models on various categories of images under several complex conditions are presented in Sect. 5. We discuss various issues and future research direction in Sect. 7 and finally conclusive remark is given in Sect. 8.
2 Global fuzzy energy-based active contour models
Krinidis and Chatzis (2009) proposed a fuzzy energy-based active contour model for image segmentation. In this section, we discuss their model and its variations.
Let \(I(X): \varOmega \rightarrow \mathfrak {R}^d\) be a given vector valued image, where \(\varOmega \subset \mathfrak {R}^2\) and \(d \ge 1\) are the image domain and the dimension of the vector I(X), respectively. In case of the gray level images, \(d=1\) while \(d=3\) for the color images. Let C be a closed contour in the image domain \(\varOmega \) which separates \(\varOmega \) into two regions: \(\varOmega _1 = \hbox {inside}\left( C \right) \) and \(\varOmega _2 = \hbox {outside}\left( C \right) \). When a given image I(X) is approximated by two regions over the image domain \(\varOmega \), the energy function is defined as
where \(c_1 \in \mathfrak {R}^d\) and \(c_2 \in \mathfrak {R}^d\) (depending on C) are the average prototypes of the regions inside and outside of the contour C, respectively. The membership function \(u\left( X \right) \in \left[ {0,1} \right] \) is the degree of membership of a pixel X for belonging to the inside of C. \(m>1\) is a weight exponent which controls the degree of ‘fuzziness’ of each membership value. L(C) is the length of the curve (C). The authors considered global image statistics for energy formulation, and it is referred to as global fuzzy energy-based active contour model.
The energy function defined in Eq. (1) will be minimized when the contour C is exactly lie on the object boundary and average prototypes \(c_1\) and \(c_2\) optimally approximate the given image I(X) into two regions, i.e., inside and outside of C, respectively. The objective is to find such a contour C which will minimize the energy function defined in Eq. (1) over all pixels \(X \in \varOmega \). Let us assume, \(C_{O}\) be the optimal extracted boundary of the object, then the energy function given in Eq. (1) will be minimized if \(C = C_O\), i.e.,
Therefore, the curve \(C_O\) is the solution to the segmentation problem.
Pseudo-level set formulationKrinidis and Chatzis (2009) defined a pseudo-level set formulation, similar to the level set method (Osher and Sethian 1988), based on the membership values u, where the curve \(C \subset \varOmega \) is implicitly represented by the pseudo-zero level set of Lipschitz similar function \(u: \varOmega \rightarrow \mathfrak {R}\), such that
Here, it is noted that the regularization or penalty term of the proposed energy function \(F(C,c_1,c_2,u)\) given in Eq. (1) is \(L(C) = \int \limits _\varOmega {\left| {\nabla H\left( {u\left( X \right) - 0.5} \right) } \right| } \mathrm{d}X\), where \(H\left( s \right) \) is a Heaviside function (Osher and Sethian 1988).
Therefore, the energy function in Eq. (1) can be rewritten as
When the energy function defined in Eq. (4) is minimized, the values of u for the pixels located inside the contour C are different from its values for the pixels located outside the contour. However, the values of u for the pixels located inside the contour C are similar. This is same for the pixels located outside the contour C.
Keeping u fixed, the authors minimized Eq. (4) with respect to \(c_{1}\) and \(c_{2}\), and these can be expressed as function of u as
and
Keeping the values of \(c_{1}\) and \(c_{2}\) fixed, the membership of each pixel, u(X), is then computed using:
For simplicity, without loss of generality, the above minimization [Eq. (7)] has been done without considering the length term (i.e., \(\mu =0\)).
Due to success of this model on segmentation of blurred, noisy and discontinuous edged images, the researchers considered this model or developed various models by modifying fuzzy energy-based active contour model (Krinidis and Chatzis 2009) in the literature to segment variety of images (Tran et al. 2014a; Wu et al. 2015; Pereira et al. 2011; Gong et al. 2015; Badshah and Ahmad 2018). Model proposed by Krinidis and Chatzis (2009) is considered to analyze liver CT images in Sajith and Hariharan (2016) and segment brain tissue in Chen et al. (2008).
Fuzzy energy-based active contour model produces good segmentation results for blurred, noisy and discontinuous edged images. However, it fails for images containing gradual tonality variations, region in-homogeneity, background clutter, object occlusion, etc. To overcome drawbacks of this model, several new methods are invented in the literature (Tran et al. 2014a; Wu et al. 2015; Pereira et al. 2011; Gong et al. 2015; Badshah and Ahmad 2018). Pereira et al. (2011) observed that fuzzy active contour model (Krinidis and Chatzis 2009) fails to properly segment images containing gradual tonality variations. To properly segment multi-tonality images, they modified fuzzy active contour model (Krinidis and Chatzis 2009) by incorporating type-2 fuzzy set which increases the fuzziness of the energy function. Instead of using u, which is type-1 fuzzy membership, they applied the type-2 values \({\tilde{u}}\) in Eqs. (5) and (6), represented by \(\tilde{c_1}\) and \(\tilde{c_2}\), respectively. The authors defined energy function as
where \({\tilde{u}}\) is type-2 membership function. The authors used four different functions (Triangular, Exponential, Logarithmic and Circular) for calculating type-2 membership values. They are defined as
Triangular:
Exponential:
Circular:
Logarithmic:
Minimization of energy function defined in Eq. (4) in fuzzy active contour model (Krinidis and Chatzis 2009) is time consuming. To minimize the computational cost of this energy function, Gong et al. proposed a novel bi-convex fuzzy variation image segmentation method in Gong et al. (2015). Bi-convex fuzzy variation function finds the global optimal solution and reduces the computational cost.
Tran et al. (2014a) developed a fuzzy energy-based active contour model with shape prior for image segmentation. The authors modified fuzzy energy function defined by Krinidis and Chatzis (2009) in Eq. (1). The modified energy function is
where \(\beta > 0\) is a weighting factor and \(F_{\mathrm{shape}}\left( u, {\hat{\psi }}\right) \) is the shape prior. The authors defined shape prior as
where \({\hat{\psi }}(x) \in [0, 1]\) is the reference shape. u as the shape indicated by a curve C which is associated with the segmentation. u can be interpreted as a binary image whose evolving shape u value is 1 for the pixels located inside C and 0 for the pixels outside C. The dissimilarity between u and \({\hat{\psi }}\) can be expressed as \(1-{\hat{\psi }}(X)\) for pixel inside C and \(\hat{\psi (X)}\) for pixel outside C. u(x) is similarly defined as in Eq. (1). The shape prior helps the model to properly segment images with background clutter and object occlusion. Limitation of this model is unavailability of shape information.
In the same direction, Pham et al. (2016) presented a fuzzy energy-based active contour model for image segmentation with shape prior based on collaborative representation of training shapes. The authors defined new energy function as
where \(S \in R\) is a coefficient vector and DS is combination of shape dictionary D. H(v) is heavy side function and \(0<T<1\). \(\beta \) and \(\mu \) are positive weighting coefficients. Here, fuzzy energy consists of a data term and a shape prior term. The data term relies on image information to guide the evolution of the contour. The prior shape is represented as the combination of atoms in the shape dictionary based on collaborative representation. Meanwhile, the shape prior term constrains the contour evolution with respect to the prior shape to handle the background clutter and the object occlusion. This model can segment images with background clutter and object occlusion even when the training set includes shapes with large variation. In addition, this shape collaborative representation model also takes less computational time compared to shape sparse representation approach (Tran et al. 2014a).
Wu et al. (2015) proposed a novel fuzzy energy-based active contour model with kernel metric for a robust and stable image segmentation. The author defined the energy function as
where K is Gaussian kernel function defined as \(K(a,b) = \exp \left( \frac{-(a-b)^2}{\sigma }\right) \) where \(\sigma \) is the bandwidth of the kernel function. Incorporation of kernel metric in the energy function and the fuzziness of the energy evolve the contour very stably without the re-initialization. This model properly segment low contrast images. In the similar direction, Badshah and Ahmad (2018) proposed a fuzzy energy-based active contour model to segment images having multi-objects with varying intensities. The authors defined energy function as
where K is kernel function and \(\mu \) is positive parameter. The authors considered Gaussian type radial basis kernel based on generalized average into their energy function. Instead of length term as in Krinidis and Chatzis (2009), Gaussian smoothing is considered as regularizer term in their energy function. Experimentally they showed that it performs well for the complex images.
3 Local fuzzy energy-based active contour models
Although the global fuzzy energy-based active models provide good segmentation results, they fail to extract proper boundary of the object when images contain noise and intensity in-homogeneity. In such cases, the local energy term derived from the local image statistics is better to extract proper object boundary. Various methods (Shyu et al. 2012b; Tran et al. 2014b; Fang et al. 2016) are invented to properly segment images under complex environment.
Shyu et al. (2012b) proposed a fuzzy energy-based active contour model involving intensity distribution information of the image to segment them. The authors defined energy function as
where Y is pixel in the local region around pixel X, I(Y) is the intensity of pixel Y, \(G_\sigma (X-Y)\) is the Gaussian kernel function and \(h_{1}(X)\) and \(h_{2}(X)\) average intensity values of local region which are obtained by Gaussian mixture model-based intensity distribution estimator operator. The intensity distributions are derived using a Gaussian mixture model-based intensity distribution estimator before the curve evolution. This model provides good segmentation results for noisy images. Tran et al. (2014b) proposed a fuzzy energy-based active contour based on local image statistics similar to Eq. (18) to extract object boundaries. In this paper, the local image intensity distribution information is derived by Hueckel operator in the neighborhood of each image pixel. The parameters of Hueckel operator are estimated by a set of orthogonal Zernike moments. Here, the fuzzy membership function is considered to measure the association degree of each image pixel to the region outside and inside the contour. This model deals with images with intensity in-homogeneity. In similar direction, Fang et al. (2016) presented a novel fuzzy region-based active contour model for image segmentation. For each pixel, local patch energy is incorporated as fuzzy energy term for the curve evolution. Due to the local information, this method is robust to the noisy images. Most of the cases, the local fuzzy energy-based methods produce better segmentation results than the global fuzzy energy-based techniques.
4 Global and local fuzzy energy-based active contour models
Several algorithms (Shyu et al. 2012a; Krinidis and Krinidis 2012; Mondal et al. 2016a, b) have been developed to better segment images with noise, region in-homogeneity, etc., by incorporating both the local and global image information into the energy function. The local energy is generated based on local images statistics which can deal with images having high intensity in-homogeneity or non-homogeneity, noise and blurred boundary or discontinuous edges. While the global term is derived from the global image statistics to avoid unsatisfactory results due to bad initialization. Combination of both local and global fuzzy energy-based active contour models provide satisfactory segmentation results.
Shyu et al. (2012a) introduced an energy function consisting of a local fuzzy energy and a global fuzzy energy terms to attract the active contour and stop it on the object boundary. They defined energy function as
where \(\sum _{i=1}^{2} \lambda _{i} \int \limits _\varOmega \left( u_{i}(X)\right) ^{m} \Vert I(X)-c_{i}\Vert ^{2} \mathrm{d}X\) is global fuzzy energy term and \(\sum _{i=1}^{2} \lambda _{i} \int \limits _\varOmega (\int \limits \left[ u_{i}(Y) \right] ^{m} G_{\sigma }(X-Y) \Vert I(Y)-f_{i}(X)\Vert \mathrm{d}Y )\mathrm{d}X\) is local fuzzy energy term. Similar to Eq. (4), \(\lambda _{1}\), \(\lambda _{2}>0\) are two fixed parameters, \(c_{1}\) and \(c_{2}\) are two constant that approximate the image intensities inside and outside of the contour (C), \(u_{1}(X) = u(X) \in [0, 1]\) is the degree of membership of I(X) to the inside of the contour C and m is a weighting exponent on each fuzzy membership. \(f_{1}(X)\) and \(f_{2}(X)\) are two local functions using to approximate the intensity means of two local regions around the pixel X inside and outside the contour C. I(Y) represents the intensities of the pixels Y which are in a local region centered at the pixel X (Y is a neighborhood of X). \(G_{\sigma }\) is a Gaussian kernel. The membership function \(u(Y)\in [0,1]\) is the belongingness of pixel Y to the inside the local region centered at the pixel X inside the contour C. \(\beta \) is a balance constant (\(0\le \beta \le 1\)). The local energy term deals with the intensity in-homogeneity presents in the images. The global energy term avoids unsatisfying results due to unsuitable initial contour position.
In this direction, Krinidis and Krinidis (2012) proposed a robust fuzzy energy-based active contour model using both global and local energy for image segmentation. The proposed energy function is defined as
\(w_{XY}\) incorporates the spatial dependency between pixel X and its neighborhood pixels Y. \(w_{XY}\) is defined as
where \(d_{XY}\) is the distance between pixels X and Y. Here, the local energy is derived based on spatial neighborhood of a pixel. This method provides better results than global fuzzy energy-based active contour model (Krinidis and Chatzis 2009).
Thieu et al. (2015) developed a fuzzy energy-based active contour model using Gaussian distribution function. They defined energy function as
\(\mu \ge 0\) controls the length |C| of the contour C. \(0<\lambda <1\) controls the influence of global and local terms in the energy function. \(c_{1}\) (\(c_{2}\)) and \(\xi _{1}\) (\(\xi _{2}\)) are, respectively, global mean and standard deviation of the Gaussian distribution inside (outside, respectively) the contour C. \(f_{1}\) (\(f_{2}\)) and \(\sigma _{1}\) (\(\sigma _{2}\)) are, respectively, the local mean and standard deviation of the Gaussian distribution inside (outside, respectively) the contour C. Y is the neighborhood pixels of X. The advantages of this model are as the local intensity allows to handle intensity in-homogeneity, while the global intensity information helps to segment images with noisy or smooth boundary.
Mondal et al. (2016a) and Mondal et al. (2016b) proposed a robust fuzzy energy-based active contour model using both global and local information. Both local spatial and gray level/color information are considered to calculate local energy. The energy function is defined as
The term I(Y) represents the intensity of a pixel Y in a neighborhood of a pixel X. \(w_{XY}\) is the weight of Yth pixel in a local neighborhood of pixel X. \(\lambda _{1}>0\) and \(\lambda _{2}>0\) are two fixed parameters. \(0\le \beta \le 1\) is a constant to control the influence of both the global energy and local energy. The weight \(w_{XY}\) depends on both the local spatial constraint and the local gray/color constraint. For each pixel \(X\in \varOmega \), the local spatial constraint reflects the damping extent of neighbors with the spatial distances from the central pixel and is defined as
with \(d_{XY}=\Vert X-Y\Vert \), where \(Y\in \eta _{X}\) is spatial neighborhood (local window) of X. The spatial constraint makes the influence of pixels within the local window. It can be changed according to their distances from the central pixel. With the help of spatial constraint, more local information is incorporated in the proposed energy model. Whereas, the local gray/color constraint is defined as
where I(X) is gray/color value of the central pixel X within a spatial local window and I(Y) is the gray/color value of Yth pixels in the same window. \(\eta _{X}\) is the neighborhood of pixel X. The denominator is a function of local density surrounding the central pixel and its value reflects the gray/color value homogeneity degree of that local window. When its value is small, the local window is more homogeneous and vice versa. This equation indicates that when the intensity value I(Y) of the Yth neighbors of central pixel X is close to I(X), \(w_{XY}^{FC}\) should be large and vice versa. The value of \(w_{XY}\) can be changed with different gray/color values of the pixels over an image and thus it reflects the damping extent in the intensity/color values. Now, the weight \(w_{XY}\) based on both the local spatial constraint and the local gray/color constraint is defined as
This model better deals with images having high intensity in-homogeneity or non-homogeneity, noise and blurred boundary or discontinuous edges due to incorporation of the local energy term in the energy function. The global energy term is used to avoid unsatisfactory results due to bad initialization. They applied their proposed model for tracking camouflaged object in Mondal et al. (2017). Due to incorporation of local and global image information into the energy function, the global and local fuzzy energy-based active contour models produce better segmentation results than the individual local and global fuzzy energy-based models.
5 Experiments
5.1 Evaluation measures
Qualities of segmentation results obtained by any algorithm depend on nature of the considered images. Here, it may be noted that one segmentation algorithm may not be always good choice for all types of images. On the contrary, different segmentation algorithms may produce different results for a particular image (Zhang et al. 2008). Therefore, performance evaluation of a segmentation algorithm is the necessary task. Qualitative comparison is the most common one. In this article, two measures namely Tanimoto coefficient/Jacard error (Krinidis and Chatzis 2009) and F-measure (Lazarevic-McManus et al. 2008) are considered to analysis the performance of these existing algorithms.
Jacard error It is defined as
where \(S_{E}\) and \(S_{O}\) are obtained and desired (ground truth) segmentation, respectively. Lower value of \(J(S_{E},S_{O})\) indicates better segmentation result.
F-measure It is defined as
where \(P_{r}\) and \(R_{e}\) are precision and recall, respectively, and defined as
Here, TP is true positive, FP is false positive and FN is false negative. Higher value of F highlights the better segmentation algorithm.
5.2 Considered images
We consider several categories of 100 imagesFootnote 1\(^{,}\)Footnote 2\(^{,}\)Footnote 3 for our experiments. Images contain intensity in-homogeneity, low contrast, background clutter, etc. To analysis the robustness of the considered algorithms under various conditions, the original images are corrupted by adding Gaussian noise, salt and pepper noise, mixture of Gaussian and salt and pepper noise and Gaussian blur. Therefore, these corrupted images are also taken into consideration for the experiment to analysis the robustness of these algorithms on difficult noisy environment.
5.3 Considered algorithms
We consider seven state-of-the-art algorithms to analysis their performance on various images under the different conditions. The selected algorithms are (i) fuzzy energy-based active contours (FEAC) (Krinidis and Chatzis 2009), (ii) novel fuzzy active contour model with kernel metric for image segmentation (NFACMKM) (Wu et al. 2015), (iii) on segmentation of images having multi-regions using Gaussian type radial basis kernel in fuzzy sets framework (FACGK) (Badshah and Ahmad 2018), (iv) localized patch-based fuzzy active contours for image segmentation (LPFAC) (Fang et al. 2016), (v) fuzzy distribution fitting energy-based active contours for image segmentation (FDFEAC) (Shyu et al. 2012b), (vi) global and local fuzzy energy-based active contours for image segmentation (GLFEAC) (Shyu et al. 2012a) and (vii) robust global and local fuzzy energy-based active contour for image segmentation (RGLFEAC) (Mondal et al. 2016a).
6 Results analysis
6.1 Segmentation of original images
Figure 1 shows few sample segmentation results using various techniques. From this figure, it is observed that due to consideration of global information, FEAC fails to properly segment images having local variations: background clutter, intensity in-homogeneity, etc (see second row of this figure). Due to incorporation of kernel metric into the energy function, both NFACMKM and FACGK produce better segmentation than FEAC. Kernel function tries to smooth the local image variation and produces better segmentation results. However, they are also fail to properly segment the images with background clutter and high region in-homogeneity, etc. Local image information-based techniques: LPFAC and FDFEAC also produce better results than FEAC. Local image information in the energy term makes both LPFAC and FDFEAC techniques robust against background clutter and region in-homogeneity. However, they are worst than both NFACMKM and FACGK. Consideration of the global and local image information in both techniques: GLFEAC and RGLFEAC help to segment images properly. Moreover, RGLFEAC segments images better than all other techniques. Table 1 displays the quantitative comparison of them. This table also highlights the similar finding.
6.2 Segmentation of blurred images
To analyze the performance of the existing algorithms under complex environment, blurred versions of the original images are also considered in this experiment. Different Gaussian functions with varying radius (\(r_{s}\)) and sigma (\(\sigma \)) are used to blur the original images. Figure 2 displays the segmentation results of blurred images. Since, Gaussian blur functions smooth the images, all these methods obtain better segmentation results than their segmentation results of corresponding original images. From this figure, it is observed that excepting RGLFEAC, all other methods produce small background patches as objects for those images having high intensity in-homogeneity. However, local and global information help RGLFEAC to properly segment images having high region in-homogeneity (Table 2).
6.3 Segmentation of corrupted images by slat and pepper noise
To analyze the robustness of these considered algorithms, we conducted an experiment to segment images which are corrupted by various amount of salt and pepper noise. Figure 3 shows the segmentation results of distorted images using different algorithms. From this figure, it is observed that methods: FEAC, LPFAC, FDFEAC and GLFEAC fail to properly segment images due to noise. However, the kernel-based methods: NFACMKM and FACGK are able to segment images along with small background patches. The local energy term defined based on region homogeneity in RGLFEAC helps to properly segment noisy images. Table 3 displays statistics of all these methods on segmentation of noisy images. This table also highlights the similar observation like visual results.
6.4 Segmentation of corrupted images by Gaussian noise
We conducted another experiment to analyze performance of these considered methods on segmentation of corrupted images by different level of Gaussian noise. Different levels of Gaussian noise are generated to corrupt the considered images. Region in-homogeneity is increased due to addition of Gaussian noise to the original images. Figure 4 presents segmentation of the images corrupted by Gaussian noise. From the figure, it is observed that FEAC is totally failed to produce proper segmentation of images under noisy environment. Due to incorporation of Gaussian kernel in the methods: NFACMKM & FACGK, they are able to obtain good segmentation results for most of corrupted images. Local fuzzy energy-based approaches: LPFAC & FDFEAC obtained good segmentation for those images having lesser density noise. RGLFEAC using local energy based on both spatial distance and gray level/color information provides the good segmentation results for most of the images. Table 4 displays statistics on performance of all the methods.
6.5 Segmentation of corrupted images by hybrid noise
To analyze the performance of the methods for segmentation of images under more complex environment, we considered the corrupted images with hybrid noise. In this experiment, mixture of different amount of Gaussian noise and salt and pepper noise are considered to corrupt the original images. We considered all these methods to segment these corrupted images. Addition of different density level of noise creates different amount of region in-homogeneity in the images and it makes segmentation task is more difficult. Figure 5 displays segmentation results of those corrupted images using various techniques. From this figure, it is observed that the global fuzzy energy-based approach: FEAC fails to segment images due to region in-homogeneity introduced by incorporation of hybrid noise. Kernel-based approaches: NFACMKM and FACGK are robust under noisy environment due to incorporation of Gaussian kernel into the energy function. Local energy-based techniques: LPFAC and FDFEAC are not robust under hybrid noisy environment. For the corrupted images where density of noise level is less, only those images they produce good results. Combination of global and local fuzzy energy-based techniques: GLFEAC and RGLFEAC are robust under hybrid noisy environment. However, GLFEAC fails to segment images with increasing density level of noise. Table 5 displays the similar finding.
7 Discussion and future work
Fuzzy energy-based active contour model proposed by Krinidis and Chatzis (2009) where energy terms are defined with global image information. This model produces good segmentation results for blurred, noisy and discontinuous edged images due to the incorporation of fuzzy logic into the function. However, it fails for images with gradual tonality variations, region in-homogeneity, background clutter, etc. Kernelized fuzzy energy active contour models (Wu et al. 2015; Badshah and Ahmad 2018) obtained stable segmentation for images with region in-homogeneity and noise due to kernel function. On the other hand, fuzzy energy-based active contour models (Fang et al. 2016; Shyu et al. 2012b) where energy terms are defined on local image information is also robust to the images having intensity in-homogeneity and noise. However, kernelized version of the global fuzzy energy-based active contour models are more robust than the local fuzzy energy-based active contour models on the level of noise and intensity in-homogeneity present in the images. Performance of the kernelized fuzzy active contour models degraded when the images contain more noise and in-homogeneous regions. In such cases, both local and global energy based active contour models (Shyu et al. 2012a; Mondal et al. 2016a) provide better segmentation results than other models. However, with increasing density level of noise, they also fail to properly segment images. From the experimental results and their analysis, we observed that the global and local fuzzy energy-based active contour models are better than all other models.
All these discussed models require the intial contour to evolve the final contour. The main drawback of these models is the requirement of manually annotated images with initial contours. One of the future direction will on automatic initialization of initial contour. Here, each pixel of the image is presented by hand-crafted features like color, texture, shape, etc. Since, features are not learned from the images, they may not robust for all types of images under the various conditions. In future, instead of considering hand-crafted features, learned feature using deep convolutional network from images will be considered. Since, all these models are working on pixel level, they need more computational time for doing segmentation of higher resolution images. Instead of pixel level, it is better to perform coarse level segmentation on super-pixel level then perform fine level segmentation on pixel level. These two step process will reduce the computational time for segmentation of high resolution images. All these models are designed for two regions segmentation (e.g., object and background). They can be extended to multi-regions segmentation.
8 Conclusions
In this article, we review existing fuzzy energy-based active contour models for image segmentation task from theoretical point of view. We also analyze robustness of all these models to segment various categories of images under various complex conditions. Findings from experiments conclude that the global image information based fuzzy active contour models fail to properly segment the images having background clutter, noise, intensity in-homogeneity. However, kernelized versions of these global models are able to produce better segmentation results than the global methods due to the effectiveness of the kernel functions. On the contrary, local fuzzy energy-based models obtain good results when density of noise level is limited. When level of noise density is increased, kernelized fuzzy energy-based active contour models as well as local fuzzy energy-based models fail to properly segment images. In such cases, the local and global fuzzy energy-based models perform well. However, with increasing density level of noise, they also fail to properly segment images. We hope that this article helps to the reader to understand fuzzy energy-based active models and several issues for image segmentation. We hope this article will help researchers toward designing new models for image segmentation.
References
Amin F, Fahmi A, Abdullah S (2019) Dealer using a new trapezoidal cubic hesitant fuzzy topsis method and application to group decision-making program. Soft Comput 23(14):5353–5366
Badshah N, Ahmad A (2018) On segmentation of images having multi-regions using Gaussian type Radial basis kernel in fuzzy sets framework. Appl Soft Comput 64:480–496
Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Kluwer Academic Publishers, Dordrecht
Cai W, Chen S, Zhang D (2007) Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation. Pattern Recognit 40(3):825–838
Caselles V, Catté F, Coll T, Dibos F (1993) A geometric model for active contours in image processing. Numer Math 66(1):1–31
Caselles V, Kimmel R, Sapiro G (1997) Geodesic active contours. Int J Comput Vis 22(1):61–79
Chan TF, Vese LA (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277
Chen Z, Qiu T, Ruan S (2008) Fuzzy adaptive level set algorithm for brain tissue segmentation. In: International conference on signal processing. IEEE, pp 1047–1050
Cohen LD, Kimmel R (1997) Global minimum for active contour models: a minimal path approach. Int J Comput Vis 24(1):57–78
Cremers D, Rousson M, Deriche R (2007) A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int J Comput Vis 72(2):195–215
Dunn JC (1973) A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters, pp 32–57
Fahmi A, Abdullah S, Amin F, Khan MSA (2019) Trapezoidal cubic fuzzy number einstein hybrid weighted averaging operators and its application to decision making. Soft Comput 23(14):5753–5783
Fang J, Liu H, Liu H, Zhang L, Liu J (2016) Localized patch-based fuzzy active contours for image segmentation. Comput Math Methods Med 2016:1–14
Fu KS, Mui J (1981) A survey on image segmentation. Pattern Recognit 13(1):3–16
Garcia-Lamont F, Cervantes J, López A, Rodriguez L (2018) Segmentation of images by color features: a survey. Neurocomputing 292:1–27
Gong M, Liang Y, Shi J, Ma W, Ma J (2013) Fuzzy c-means clustering with local information and kernel metric for image segmentation. IEEE Trans Image Process 22(2):573–584
Gong M, Tian D, Su L, Jiao L (2015) An efficient bi-convex fuzzy variational image segmentation method. Inf Sci 293:351–369
Gonzalez RF, Woods RE (2008) Digital image processing. Pearson Education, Singapore
Gunn SR, Nixon MS (1997) A robust Snake implementation; a dual active contour. IEEE Trans Pattern Anal Mach Intell 19(1):63–68
Kass M, Witkin A, Terzopoulos D (1988) Snakes: active contour models. Int J Comput Vis 1(4):321–331
Khan MW (2014) A survey: image segmentation techniques. Int J Future Comput Commun 3(2):89
Krinidis S, Chatzis V (2009) Fuzzy energy-based active contours. IEEE Trans Image Process 18(12):2747–2755
Krinidis S, Chatzis V (2010) A robust fuzzy local information c-means clustering algorithm. IEEE Trans Image Process 19(5):1328–1337
Krinidis S, Krinidis M (2012) Fuzzy energy-based active contours exploiting local information. In: International conference on artificial intelligence applications and innovations. Springer, pp 175–184
Lankton S, Tannenbaum A (2008) Localizing region-based active contours. IEEE Trans Image Process 17(11):2029–2039
Lazarevic-McManus N, Renno JR, Makris D, Jones GA (2008) An object-based comparative methodology for motion detection based on the F-measure. Comput Vis Image Underst 111(1):74–85
Li C, Kao CY, Gore JC, Ding Z (2008) Minimization of region-scalable fitting energy for image segmentation. IEEE Trans Image Process 17(10):1940–1949
Li C, Xu C, Gui C, Fox MD (2010) Distance regularized level set evolution and its application to image segmentation. IEEE Trans Image Process 19(12):3243–3254
Liu G, Zhang Y, Wang A (2015) Incorporating adaptive local information into fuzzy clustering for image segmentation. IEEE Trans Image Process 24(11):3990–4000
Malladi R, Sethian JA, Vemuri BC (1995) Shape modeling with front propagation: a level set approach. IEEE Trans Pattern Anal Mach Intell 17(2):158–175
Melin P (2018) Type-2 fuzzy logic in pattern recognition applications. In: Type-2 fuzzy logic and systems. Springer, pp 89–104
Mitchell HB (2005) Pattern recognition using type-II fuzzy sets. Inf Sci 170(2):409–18
Mondal A, Ghosh S, Ghosh A (2016) Robust global and local fuzzy energy based active contour for image segmentation. Appl Soft Comput 47:191–215
Mondal A, Murthy KR, Ghosh A, Ghosh S (2016) Robust image segmentation using global and local fuzzy energy based active contour. In: IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1341–1348
Mondal A, Ghosh S, Ghosh A (2017) Partially camouflaged object tracking using modified probabilistic neural network and fuzzy energy based active contour. Int J Comput Vis 122(1):116–148
Naz S, Majeed H, Irshad H (2010) Image segmentation using fuzzy clustering: a survey. In: International conference on emerging technologies (ICET). IEEE, pp 181–186
Nguyen TNA, Cai J, Zhang J, Zheng J (2012) Robust interactive image segmentation using convex active contours. IEEE Trans Image Process 21(8):3734–3743
Osher S, Sethian JA (1988) Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J Comput Phys 79(1):12–49
Pal NR, Pal SK (1993) A review on image segmentation techniques. Pattern Recognit 26(9):1277–1294
Pereira CL, Bastos CA, Ren TI, Cavalcanti GD (2011) Fuzzy active contour models. In: International conference on fuzzy systems. IEEE, pp 1621–1627
Pham VT, Tran TT, Shyu KK, Lin C, Wang PC, Lo MT (2016) Shape collaborative representation with fuzzy energy based active contour model. Eng Appl Artif Intell 56:60–74
Sahoo PK, Soltani S, Wong AK (1988) A survey of thresholding techniques. Comput Vis Graph Image Process 41(2):233–260
Sajith A, Hariharan S (2016) A region based active contour approach for Liver CT image analysis based on fuzzy energy minimization 7(5):779–785
Shyu KK, Pham VT, Tran TT, Lee PL (2012a) Global and local fuzzy energy-based active contours for image segmentation. Nonlinear Dyn 67(2):1559–1578
Shyu KK, Tran TT, Pham VT, Lee PL, Shang LJ (2012b) Fuzzy distribution fitting energy-based active contours for image segmentation. Nonlinear Dyn 69(1–2):295–312
Thieu QT, Luong M, Rocchisani JM, Sirakov NM, Viennet E (2015) Efficient segmentation with the convex local-global fuzzy Gaussian distribution active contour for medical applications. Ann Math Artif Intell 75(12):249–266
Tran TT, Pham VT, Shyu KK (2014a) Image segmentation using fuzzy energy-based active contour with shape prior. J Vis Commun Image Represent 25(7):1732–1745
Tran TT, Pham VT, Shyu KK (2014b) Zernike moment and local distribution fitting fuzzy energy-based active contours for image segmentation. SIViP 8(1):11–25
Wang L, Chang Y, Wang H, Wu Z, Pu J, Yang X (2017) An active contour model based on local fitted images for image segmentation. Inf Sci 418:61–73
Wu Y, Ma W, Gong M, Li H, Jiao L (2015) Novel fuzzy active contour model with kernel metric for image segmentation. Appl Soft Comput 34:301–311
Yezzi A, Kichenassamy S, Kumar A, Olver P, Tannenbaum A (1997) A geometric Snake model for segmentation of medical imagery. IEEE Trans Med Imaging 16(2):199–209
Zaitoun NM, Aqel MJ (2015) Survey on image segmentation techniques. Procedia Comput Sci 65:797–806
Zhang H, Fritts JE, Goldman SA (2008) Image segmentation evaluation: a survey of unsupervised methods. Comput Vis Image Underst 110(2):260–280
Zhang K, Song H, Zhang L (2010a) Active contours driven by local image fitting energy. Pattern Recognit 43(4):1199–1206
Zhang K, Zhang L, Song H, Zhou W (2010b) Active contours with selective local or global segmentation: a new formulation and level set method. Image Vis Comput 28(4):668–676
Zhang X, Sun Y, Wang G, Guo Q, Zhang C, Chen B (2017) Improved fuzzy clustering algorithm with non-local information for image segmentation. Multimed Tools Appl 76(6):7869–7895
Zhang K, Zhang L, Lam KM, Zhang D (2013) A local active contour model for image segmentation with intensity inhomogeneity. arXiv preprint arXiv:1305.7053
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Ethical approval
This article does no contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mondal, A., Ghosh, K. State-of-the-art fuzzy active contour models for image segmentation. Soft Comput 24, 14411–14427 (2020). https://doi.org/10.1007/s00500-020-04794-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-04794-y