A new effective hybrid segmentation method based on C–V and LGDF

Abstract

Image segmentation is a significant research topic in image processing and computer vision. Active contour methods (ACMs) are widely used in image segmentation. In this paper, a new hybrid ACM segmentation model based on Chan–Vese (C–V) and Local Gaussian Distribution Fitting (LGDF) methods is proposed for the images with intensity inhomogeneity. In this model, new gradient descent flow equations are proposed and applied for the energy minimization of C–V and LGDF methods. Firstly, the proposed C–V method is applied to the image to effectively and quickly find the homogeneous regions of the image. Then, the proposed LGDF method is performed in these regions to detect inhomogeneous areas of the image. Thus, more effective and successful segmentation is obtained for inhomogeneous images. Experimental results show that the satisfactory segmentation results have been obtained by the proposed method for MRI and real images. Also, the proposed method is compared with the local binary fitting, LGDF, adaptive local-fitting-based, global and local weighted signed pressure ACMs, and convolutional neural network-based methods.

1 Introduction

Image segmentation is the process of separating an image into meaningful areas based on similar properties such as intensity, color and texture. Segmentation is used in image processing application areas such as medical imaging, satellite imaging and industrial vision systems for the purpose of image analysis, target extraction, and recognition.

The active contour method ACM is a widely used for image segmentation process in recent years [1,2,3,4,5,6,7]. ACM obtains successful segmentation results due to the ability of handle structural changes. ACMs can be categorized into two classes, such as edge-based method [8,9,10] and region-based methods [11,12,13,14,15,16,17,18,19]. Edge-based ACMs utilize the gradient information of the image. These methods are more sensitive to noise and prone to a local minimum [8]. Region-based ACMs use internal energy and external energy information of the image. Therefore, these methods are less sensitive to noise.

Region-based ACMs can be classified into global information [11, 12], local information [13,14,15,16,17,18] and hybrid information-based ACMs [19,20,21,22,23]. Global information-based ACMs direct the contour to detect the boundaries of the object in the image using the inner and outer area informations. These methods are not very successful for the images with intensity inhomogeneity. Local information-based ACMs use the local difference information of the foreground and background regions of the image. Local and global information-based ACMs are less sensitive to noise and perform better results for the images with weak edges. However, the success of these methods depends on the initialization contour. Some of the advantages and disadvantages of the current Region-based ACMs are summarized in Table 1.

In this paper, to obtain a more effective segmentation a novel hybrid ACM based on Chan-Vese (C–V) and Local Gaussian Distribution Fitting (LGDF) methods using new gradient descent flow equations is proposed. In this method, firstly, the proposed C–V method is performed to the image to quickly and effectively detect the homogeneous areas. Then, the proposed LGDF method is applied into these areas to detect inhomogeneous regions of the images. The proposed hybrid method has provided effective and highly accurate segmentation.

The main contributions of this paper are as follows: (1) we introduce a novel C–V and LGDF-based hybrid ACM using new gradient descent flow equations for inhomogeneous images; (2) we remove the initial contour position problem; (3) we achieve faster and more effective results than the existing methods.

Table 1 Brief advantages and disadvantages of region-based ACMs

The organization of the paper is as follows: Literature reviews are given in Sect. 2. In Sect. 3, preliminary information about the applied methods is described. In Sect. 4, the proposed hybrid ACM is explained. The experimental results and analysis are demonstrated in Sect. 5. The conclusion is given in Sect. 6.

2 Literature review

In recent years, researchers have developed ACM-based segmentation methods. Chan and Vese proposed a global information-based ACM method namely, C–V method, [11], for homogeneous images. Li et al. [13] suggested local binary fitting (LBF) method for inhomogeneous intensity images by using local intensity means and the Gauss kernel function. Wang at al. [14] presented the LGDF method for inhomogeneous images by using the Gaussian distribution with local means and variances. Ma et al. [6] proposed an adaptive local fitting (ALF) method by using adaptive local fitting and regularization energies to drive the initial contour to the object boundary.

Hybrid methods have also been developed by the researchers using region-based or edge-based methods. Zhang et al. [19] suggested selective binary and Gaussian filtering regularized level set method by using the edge-based and region-based ACMs. Wang et al. [20] proposed local and global information-based ACM by using Gaussian distribution for intensity inhomogeneity images. Zhang et al. [21] suggested a hybrid signed pressure force (SPF) function using the local and global information of the image to reduce the effect of the initialization contour. Han et al. [22] proposed a hybrid ACM by using global and local weighted SPF (GLWSPF). Peng et al. [15] suggested a hybrid ACM, in which the intensity distribution of each region was assumed as a Gaussian distribution with spatially varying mean and variance. Vakili et al. [23] proposed a two-stage segmentation method based on mixture of Gaussian distribution. Soomro et al. [24] presented a two-stage hybrid ACM by using global region force and geodesic edge terms. Wang et al. [16] presented a hybrid ACM based on LBF and the local image fitting methods. Zhao et al. [25] proposed an ACM by using local and global Gaussian distribution methods.

Moreover, convolutional neural network (CNN) architectures have also been used for image segmentation in recent years [26,27,28]. Long et al. [29] suggested fully convolutional networks (FCNs) for semantic segmentation. Ronneberger et al. [30] presented U-Net method based on FCNs. He et al. [31] proposed Mask R-CNN to tackle pixel-wise object instance segmentation. Chen et al. [32] offered an efficient CNN-based segmentation method, namely DeepLab model, which includes Atrous convolutions, a deep convolutional network and Atrous spatial pyramid pooling components. Chen et al. [33] improved their previous CNN-based segmentation work as DeepLabv3 system to capture multiscale context.

Most of the CNN-based segmentation methods fail to find details of object boundaries [6, 34]. In addition, CNN-based segmentation methods require training data and more runtime than the ACM-based segmentation methods [6, 35]. On the other hand, hybrid methods have been developed by using SPF functions or edge-based methods which are faster and simple. Also, in hybrid segmentation methods, Gaussian distribution is commonly used. However, these methods are computationally complex and so their runtime is high. Besides, the initial contour has a high impact on the success of these methods. The C–V is an effective global segmentation ACM. The LGDF is a successful local segmentation ACM in determining the details of the images by using the mean and variance information. In this paper, by considering the advantages of these methods, a new hybrid ACM has been proposed. Using the new gradient descent flow equations in the proposed method provides a fast, effective, and detailed segmentation. Also, the method has eliminated the initial contour position problem.

3 Preliminaries

3.1 Chen-Vese (C–V) ACM

C–V is a successful region-based ACM for homogeneity images [11]. In this method, in order to determine the region boundary, the energy functional is minimized as follows:

$$\begin{aligned} \begin{aligned}&E^{\text {CV}}(c_1,c_2,C)=\mu {\text {Length (C)}} + \nu {\text {Area (inside(C))}} \\&\quad +\lambda _1\int _{{\text {inside(C)}}}(I(x)-c_1)^2 H(\phi (x)){\text {d}}x\\&\quad +\lambda _2\int _{{\text {outside(C)}}}(I(x)-c_2)^2 (1-H(\phi (x))){\text {d}}x \end{aligned} \end{aligned}$$

(1)

where inside (C) and outside(C) denote the inside and outside regions of the contour C, respectively. The detailed definition of all the parameters is found in [11]. The gradient descent flow equation is given as follows [11]:

$$\begin{aligned} \begin{aligned}&\frac{\partial \phi (x,t)}{\partial t} =\\&\quad \delta (\phi ) \left[ -\lambda _1 (I(x)-c_1)^2 +\lambda _2 (I(x)-c_2)^2 + \mu {\text {div}}\left( \frac{\nabla \phi }{|\nabla \phi |}\right) -\upsilon \right] \end{aligned} \end{aligned}$$

(2)

The C–V method can effectively detect boundaries of objects without using the gradient of the image. It also provides successful segmentation for noisy images. However, because of using the global intensity information of the image, the C–V method may fail for inhomogeneous images [18].

3.2 Local Gaussian distribution fitting (LGDF) ACM

LGDF is a region-based ACM, in which local image intensities are defined by Gaussian distributions with different means and variances [14]. The energy formulation of the LGDF method is defined as follows [14]:

$$\begin{aligned} \begin{aligned}&E^{\text {LGDF}}_x=\\&\quad \int _\varOmega \left( \sum _{i=1}^{N} \int _\varOmega \omega (x-y) \left[ \log (\sigma _i(x))+\frac{\left( I(y)-u_i(x) \right) ^2 }{2\sigma _i(x)^2} \right] {\text {d}}y \right) {\text {d}}x \end{aligned} \end{aligned}$$

(3)

where $u_i (x)$ and $\sigma _i (x)$ are local intensity means and standard deviations, respectively. $\omega (x-y)$ is a Gaussian kernel function and the detailed definition of all the parameters can be found in [14]. The gradient descent flow equation is given as follows [14]:

$$\begin{aligned} \frac{\partial \phi }{\partial t}= & {} - \delta (\phi ) (e_1 - e_2) + \nu \delta (\phi ){\text {div}}\left( \frac{\nabla \phi }{|\nabla \phi |}\right) \nonumber \\&+ \mu \left( \nabla ^2 \phi -{\text {div}}\left( \frac{\nabla \phi }{|\nabla \phi |}\right) \right) \end{aligned}$$

(4)

where, $e_1$ and $e_2$ are variables formulated using local mean intensities and variances [14]. ($-\delta (\phi ) (e_1 - e_2)$), ($\nu \delta (\phi ){\text {div}}\left( \frac{\nabla \phi }{|\nabla \phi |}\right)$), and ($\mu \left( \nabla ^2 \phi -{\text {div}}\left( \frac{\nabla \phi }{|\nabla \phi |}\right) \right)$) are the data fitting, the arc length, and level set regularization terms, respectively. The detail of the terms are described in [14] [36] The method is less sensitive to the initial contour position.

4 Proposed method

A new hybrid ACM which consist of two stages is proposed for inhomogeneous images. In the first stage, the proposed C–V effectively and quickly segments the homogeneous regions in the image. In the second stage, the proposed LGDF method is applied to the image regions obtained in the first stage to determine inhomogeneity regions.

4.1 First stage: proposed global region-based ACM

The proposed global region-based method, which eliminates the initial contour position problem, is based on the C–V method. The energy formulation of the proposed C–V (PCV) method is defined as follows:

$$\begin{aligned} \begin{aligned}&E_{\text {PCV}}(c_1,c_2,C)=\\&\quad +\int _{{\text {inside(C)}}}(I(x)-c_1)^2 H(\phi (x)){\text {d}}x\\&\quad +\int _{{\text {outside(C)}}}(I(x)-c_2)^2 (1-H(\phi (x))){\text {d}}x \end{aligned} \end{aligned}$$

(5)

where I(x) is the image, $c_1$ and $c_2$ are the inside and outside average intensities of the contour C, respectively, and are given as follows:

$$\begin{aligned} {\left\{ \begin{array}{ll} c_1=\frac{\int _\varOmega I(x) H(\phi (x)){\text {d}}x}{H(\phi (x)){\text {d}}x} \\ c_2=\frac{\int _\varOmega I(x) (1-H(\phi (x))){\text {d}}x}{(1-H(\phi (x))){\text {d}}x} \end{array}\right. } \end{aligned}$$

(6)

where $\varOmega$ is the image domain, $H(\phi (x))$ is the regularized Heaviside function which is defined as below:

$$\begin{aligned} H_ \varepsilon (\phi )=\frac{1}{2}\left( 1+\frac{2}{\pi } {\text {arctan}}\left( \frac{\phi }{\varepsilon } \right) \right) \end{aligned}$$

(7)

where $\varepsilon$ is a small constant, $\phi$ is the zero level set of a Lipschitz function. The detail of the term is described in [11]. The new gradient descent flow equation of PCV is defined as:

$$\begin{aligned} \frac{\partial \phi }{\partial t}= & {} {} \frac{\mu div \left( \frac{\nabla \phi }{| \nabla \phi |}\right) }{\max \left( \max \left| {\text{ div }}\left( \frac{\nabla \phi }{|\nabla \phi |}\right) \right| \right) } \nonumber \\&+\frac{1}{- (I(x)-c_1)^2+(I(x)- c_2)^2} \end{aligned}$$

(8)

where $\mu {\text {div}}\left( \frac{\nabla \phi }{|\nabla \phi |}\right)$ term is necessary to maintain the regularity of the contour, which means that the smoothness over the region boundaries, and it performs shortening or smoothing effect on the contour [36].

The C–V method has an initial contour position problem. The PCV method eliminates this problem and is successful in determining the region boundary regardless of the size and position of the initial contour. A comparison of the PCV and the C–V is given in Fig. 1. It is evident from the figure that the PCV method segments more effectively and faster than the C–V method.

The basic steps of the proposed global region-based method are as follows:

• 1. Get the image, I, and the initial contour, C.

• 2. Calculate image intensities, $c_1$ and $c_2$, by using Eq. 6.

• 3. Calculate the gradient descent flow by using Eq. 8.

• 4. Control the contour evolution. If is not stable, repeat steps 2–4.

The PCV provides effective global region segmentation. The obtained global region is used as the initial contour position of the second stage.

Fig. 1

Comparison of the PCV and C–V methods in terms of initial contour position effect: a Image, b Large/small initial contour, c C–V and iterations, d PCV and iterations

4.2 Second stage: proposed local region-based ACM

The proposed local region-based method is based on the LGDF method which uses both the mean and variance information of the image. The energy formulation of the proposed LGDF (PLGDF) method is defined as follow:

$$\begin{aligned} \begin{aligned}&E^{\text {PLGDF}}\left( u_1(x),u_2(x),\sigma ^2_1(x),\sigma ^2_2(x)\right) =\\&\qquad -\int _{\varOmega {\text {inside}}}\omega (x-y) \log p_{1,x}\left( I(y)\right) {\text {d}}y\\&\qquad -\int _{\varOmega {\text {outside}}}\omega (x-y) \log p_{2,x}\left( I(y)\right) {\text {d}}y \end{aligned} \end{aligned}$$

(9)

where I(y) is the image. $u_1(x)$ and $\sigma ^2_1(x)$ are the local intensity and standard deviations, respectively [14]. $p_{1,x}\left( I(y)\right)$ is the probability intensity and is defined as:

$$\begin{aligned}&p_{1,x}\left( I(y)\right) =\frac{1}{\sqrt{2\pi }\sigma _i} exp\left( -\frac{\left( u_i(x)-I(y) \right) ^2}{2\sigma _i(x)^2} \right) \end{aligned}$$

(10)

$$\begin{aligned}&\quad {\left\{ \begin{array}{ll} u_1(x)=\frac{\int \omega (x-y) I(y) H(\phi (x)){\text {d}}y}{\int \omega (x-y) H(\phi (x)){\text {d}}y} \\ u_2(x)=\frac{\int \omega (x-y) I(y) \left( 1-H(\phi (x)) \right) {\text {d}}y}{\int \omega (x-y)\left( 1-H(\phi (x)) \right) {\text {d}}y} \end{array}\right. } \end{aligned}$$

(11)

$$\begin{aligned}&\quad {\left\{ \begin{array}{ll} \sigma _1(x)^2=\frac{\int \omega (x-y) \left( u_1(x)-I(y) \right) ^2 H(\phi (x)){\text {d}}y}{\int \omega (x-y) H(\phi (x)){\text {d}}y} \\ \sigma _2(x)^2=\frac{\int \omega (x-y) \left( u_2(x)-I(y) \right) ^2 \left( 1-H(\phi (x)) \right) {\text {d}}y}{\int \omega (x-y)\left( 1-H(\phi (x)) \right) {\text {d}}y} \end{array}\right. } \end{aligned}$$

(12)

The new gradient descent flow equation of PLGDF is given as follows:

$$\begin{aligned} \frac{\partial \phi }{\partial t} = -\delta (\phi ) (e_1 - e_2)+ \nu \delta (\phi ) + \mu \left( \nabla ^2 \phi \right) \end{aligned}$$

(13)

where $\delta$ is the derivative of Heaviside function. $e_1$ and $e_2$ are the variables formulated using local mean intensities and variances as follows [14]:

$$\begin{aligned} e_i(x)= \int _{\varOmega }\omega (x-y)\left[ \log \sigma _i(x) + \left( -\frac{\left( u_i(x)-I(y) \right) ^2}{2\sigma _i(x)^2} \right) \right] {\text {d}}y \end{aligned}$$

(14)

The basic steps of the proposed local region-based method are as follows:

• 1. Get the image, I, and the initial contour result of Stage 1.

• 2. Calculate the mean functions, $u_1(x)$ and $u_2(x)$, by using Eq. 11.

• 3. Calculate the variance functions, $\sigma _1(x)$ and $\sigma _2(x)$, by using Eq. 12.

• 4. Calculate the gradient descent flow by using Eq. 13.

• 5. Control the contour evolution. If is not stable, repeat steps 2–5.

The segmentation process of the PLGDF method is shown in Fig. 2. The global region segmentation result of Stage 1 is used as the initial contour. The PLGDF provides faster and more effective segmentation as it has an initial contour away from the insignificant regions. The flowchart of the proposed hybrid method is shown in Fig. 3.

Fig. 2

Segmentation process of the PLGDF, a image, b initial contour, c segmentation, d obtained region

Fig. 3

Flowchart of the proposed hybrid method

5 Experimental results and discussion

This section presents a comparative study of the proposed method with LBF [13], LGDF [14], ALF [6], GLWSPF [22] ACMs and CNN-based methods such as Deeplabv3-ResNet101 [33] and Mask R-CNN [31]. All the experiments have been performed on a computer with a CPU Intel Core i5 2.5 GHz and 4 GB RAM by using MATLAB R2018a.

5.1 Segmentation results for MR images

We have used different T1-MR brain images from the BrainWeb dataset [37] to demonstrate the success of the proposed model. The noise level is 3%, and intensity inhomogeneity is 20% for all images [6]. The comparative segmentation results are shown in Fig. 4. It can be seen from the figure that the LBF, LGDF, and ALF methods have not provided detailed segmentation. Incomplete segmentation results have been obtained by the LBF method in the left and right middle region of the first image and the upper left region of the fourth image. Similar results have occured in the upper left part of the third image with the LGDF method and in the right middle region of the first image with the ALF method. The hybrid GLWSPF method has obtained incomplete results in the upper left regions of the third and fifth images. Also, GLWSPF has caused noise. As can be seen from the figure that the proposed hybrid method has effectively segmented all regions of the image.

Fig. 4

Segmentation results of the LBF, LGDF, GLWSPF, ALF and the proposed method for brain MRIs

5.2 Segmentation results for real images

Bird, Aircraft, and Pot images from the Berkeley segmentation data set [38] are used to evaluate the performance of the proposed method for real images. Figure5 shows the comparative segmentation results. The LBF method has not detected the branch details in the middle–lower region and the right region of the Bird image. It has determined the difference in brightness on the left branch region and intensity on the bird region. Also, the LBF method has achieved the detail in the tail region of the aircraft image. However, it has performed incorrect segmentation in the fuselage. Moreover, for the Pot image, this method has found the pot region, but it has not completely obtained the details in the region. The LGDF method has performed more detailed segmentation in the branch region of the Bird image than the LBF method. However, it has not detected the branch details in the left region. The LGDF method has successfully determined the region of the aircraft image. However, it has not completely obtained the details in the tail and fuselage regions. Also, the LGDF method has determined the pot region, but it has not obtained the details of the Pot image. The GLWSPF method has performed rough segmentations for the Bird and Pot images. It has not revealed any details about the region in these images. Also, the GLWSPF method has provided good segmentation results for the aircraft image. It has found the ’A’ icon in the tail region and star icon in the fuselage region. However, GLWSPF has caused noise. It is evident from the figure that the proposed hybrid method has provided the best segmentation results for each image. The method has accurately segmented all the regions of the images. Moreover, the proposed method has revealed all the details on the images.

Fig. 5

Segmentation results of the LBF, LGDF, GLWSPF, and the proposed method for real images

In order to compare the proposed method with the CNN-based methods Aircraft, Eagle, and Duck images from the Berkeley segmentation data set [38] are used. Deeplabv3-ResNet101 [33] and Mask R-CNN [31] methods have been used in comparison. Deeplabv3-ResNet101 has been constructed by a Deeplabv3 model with a ResNet-101 backbone [33]. These methods have been trained on a subset of COCO train2017 set. The comparative segmentation results are shown in Fig. 6. The Deeplabv3-ResNet101 method has found the object regions in the images. However, the method has not obtained the aircraft details. The Mask R-CNN method has found the object regions of the Aircraft and Duck images. However, it has not detected the small eagle in the lower region of the Eagle image. As can be seen from the figure that the proposed method has provided detailed segmentation results for the Aircraft and Eagle images. Our method has revealed all the details on these images. However, excessive segmentation has occured in some regions of the Duck image with the proposed method.

Fig. 6

Segmentation results of the Deeplabv3, Mask R-CNN, and the proposed method for real images

5.3 Performance analysis of the segmentation results

A comparative performance analysis of the results have been investigated by quality evaluation indexes, such as Jaccard similarity (JCS) rate, Dice coefficient (DC) rate, runtime (sec), and iterations (Iter). The parameters of JCS and DC rates are defined as [14]:

$$\begin{aligned} {\text {JCS}}=\frac{\left| I_s \cap I_g\right| }{\left| I_s \cup I_g\right| } {\text {DC}}=\frac{2\left| I_s \cap I_g\right| }{\left| I_s \right| + \left| I_g\right| } \end{aligned}$$

(15)

where $I_s$ and $I_g$ are the segmentation result and ground truth of the image, respectively. High JCS and DC values indicate the success of the segmentation.Table 2 presents the performance comparison results for MRIs. LBF and LGDF methods have very close DC and JCS results. LGDF is the method with the highest number of iterations. The GLWSPF method is the fastest in terms of the number of iterations and runtime. The ALF is the slowest method in terms of runtime. It is clear from the table that the proposed method has the highest JCS and DC values for all the MRIs except MRI-1. Our method is faster than the LBF, LGDF, and ALF methods. Also, the proposed method has obtained acceptable iteration number although it performs both global and local segmentation.

Table 2 Performance comparisons of the proposed method with the ACMs for MRIs

The performance comparison results of the ACMs for real images are given in Table 3. The GLWSPF method has provided the best results in terms of iterations and runtime. However, it has not obtained high values of JCS and DC. As can be seen from the table that the proposed method has provided the best JCS and DC values for the Bird and Aircraft images. For the Pot image, the LGDF method has the highest JCS and DC values. However, the LGDF is slowest method in terms of iterations and runtime.

Table 3 Performance comparisons of the proposed method with the ACMs for real images

Table 4 shows the performance comparison of the proposed method with the CNN-based methods. The proposed method has obtained the highest JCS and DC values for the Aircraft and Eagle images. For the Duck image, although our method has found the object region of the image, it has obtained lower JCS and DC values than the CNN-based methods. However, the CNN-based methods are computationally complex and require training data to learn segmentation.

Table 4 Performance comparisons of the proposed method with the CNN-based methods for real images

6 Conclusion

In this paper, a novel hybrid ACM based on PCV and PLGDF methods using new gradient descent flow equations has been presented for inhomogeneity images. In this method, firstly, the PCV method has been performed to the image to quickly and successfully detect the homogeneous regions. Then, the PLGDF method has been applied into these areas to detect inhomogeneous regions of the images. Experimental results illustrate that the proposed hybrid method provides efficient, accurate, and fast segmentation for various types of images. Also, this method has eliminated the initial contour position problem of ACMs. Moreover, a comparative study has been performed on the MR and real images. The proposed method has achieved faster and more effective results than some state-of-the-art methods. As a future work, the new gradient descent flow equations can be used on other ACMs. Also, the proposed methods can be combined with CNN methods.