Color Texture Image Segmentation Using Histogram-Based CV Model Driven by Local Contrast Pattern

Abstract

This paper proposes a novel texture descriptor called local contrast pattern (LCP) to drive a histogram-based Chan-Vese(CV) model for color texture image segmentation. The LCP has two features: differential contrast and orientation. The former measures the variation of local intensity and the latter extracts the texture orientation information. In order to enhance the localization, a truncated Gaussian kernel function is also incorporated that determines a positive correlation between weight and distance. Then, a novel histogram-based CV model is established which is guided by a combination of the LCP feature maps and the kernel to obtain color texture segmentation. The effectiveness of the LCP descriptor in color texture segmentation was prove to be true by many single variable validation experiments. Comparisons with many color-texture image segmentation models demonstrate that our proposed model can not only successfully partition all types of images but also bear strong robustness for illumination, noise and curve initialization.

1 Introduction

The existing active contour models can be roughly categorized into two classes: edge-based models [12, 22] and region-based models [1, 6, 8, 18]. The edge-based ones usually use a gradient-based boundary detection function to stop contours for the real edges. Many new algorithms are derived from the local region based models because they could catch more details of objects. Chunming Li et al. proposed the Local Binary Fitting (LBF) model [10, 11] by generating the variances in a small local region whose size was limited by a proper kernel function. Since its locality, the model turned out to deal with intensity inhomogeneity effectively. Now many new algorithms are derived from the local region based models because they could catch more details of objects. Some researchers also combine the information of global and local regions to construct driving energy [2, 7, 20]. Besides, They try to extract the object shape as a priori to eliminate the influence of other unwanted ones [4, 13, 14, 19, 24]. However, the methods above are invalid for color texture image segment, in addition, uneven illumination and all kinds of noises are inevitable.

In this paper, a novel histogram-based CV model driven by local contrast pattern (LCP) and restrainted by a kernel function is proposed. It aims to solve color texture image segmentation problem even in some extreme circumstances like uneven illumination, noise and multifarious initialization. There are two main contributions: firstly, the LCP descriptor is newly proposed, by calculating the local intensity changes and the gradient orientation to describe texture structure correctly. It can not only eliminate inhomogeneous intensity but also distinguish different texture types. Secondly, we incorporate the LCP descriptor into a CV model to segment color texture image by calculating statistical histograms. In order to maintain better locality, a truncated Gaussian kernel function is used to suppress undesirable clusters in global domain and smooth the curves. Experiments demonstrate the proposed scheme not only achieves effective segmentation results but also has a good robustness for illumination, noise and curve initialization on color texture image.

2 Related Works

2.1 Local Histogram-Based (LH) Model

The local histogram-based active contour model [15] can be applied to segment texture image. It make a feature transformation by calculating statistic histograms in a neighbourhood of each pixel, therefore, a histogram denotes a pixel value instead of its intensity. In fact, it is a deformation process of evolving curve by minimizing the distance between the object histogram and the average one in the CV model. Because of the statistical property of histogram, its robustness for noise and complex intensity variation is better than the traditional CV model. Though mediocre, the LH model is a powerful carrier to fusion other features to segment image.

Given a gray-scale image $I:\varOmega \rightarrow [0,L]$ (L usually takes 255), $\varOmega$ represents the image domain and the evolving contour C divides I into two regions denoted by in(C) and out(C). Let $N_{x,r}$ be a local patch centered at pixel $x\in \varOmega$ with radius r, for $0\le \alpha \le L$, the histogram and corresponding cumulative distribution function of x are listed below:

$$\begin{aligned} P_r^x(\alpha )=\frac{|\{z\in N_r^x\cap \varOmega :I(z)=\alpha \}|}{|N_r^x\cap \varOmega |}, \end{aligned}$$

(1)

$$\begin{aligned} F_r^x(\alpha )=\frac{|\{z\in N_r^x\cap \varOmega :I(z)\le \alpha \}|}{|N_r^x\cap \varOmega |}. \end{aligned}$$

(2)

Based on the definitions above, the energy function is defined by:

$$\begin{aligned} \begin{aligned} E_{LH}(P_i,P_o,C)&= \lambda _1\int _{in(C)}W_1(P_i,P_r^x)dx\\&+\lambda _2\int _{out(C)}W_1(P_o,P_r^x)dx+ \mu \int _Cds, \end{aligned} \end{aligned}$$

(3)

where $\lambda _1$, $\lambda _2$ and $\mu$ are constants, $P_i^x$ and $P_o^x$ are the averages of histograms at all pixels inside and outside the curve C. $\mu \int _Cds$ computes the perimeter of the curve as a constraining force to reduces redundant curves, and $W_1(\cdot )$ is the linear Wasserstein distance which compares two normalized histograms, its computation formula is shown as follows:

$$\begin{aligned} W_1(P_1,P_2) = \int _0^L|F_1(y) - F_2(y)|dy. \end{aligned}$$

(4)

It is obvious that the energy function consists of two parts: a region-based fitting energy term and a length penalty term. The former term compares the histograms aiming at finding a best segmentation curve so that the pixel histogram can optimally fit the average histograms of the foreground or background. It is a crucial image force guiding the active curves to reach the real boundaries. While the latter term calculates the contour perimeter and smooths the curve. The LH model is the earliest exploration to carry histogram to generate a novel thinking on texture segmentation.

2.2 LBP-Driven Vector-Valued Chan-Vese (LBPCV) Model

The LBP-driven vector-valued Chan-Vese (LBPCV) model [17] is a histogram based vector CV model. It combines with many characteristics of the image through histogram as vector filters for texture image segmentation, like intensity, first order derivatives and especially the local binary pattern (LBP) values. From this model, one can also derive some other versions by altering the filter channels.

The LBP operator, which can extract texture features in a local region, is first proposed by Ojala et al. [16]. It has many good properties such as simplicity, illumination and rotation invariance, so it has been widely utilized in face recognition [3] and texture classification [16]. The original LBP operator is defines in a $3\times 3$ local window centered at a pixel x (see Fig. 1), then it binarizes the intensity differences between the center pixel and its neighbors and encodes it into a binary number. Its computation formula is presented as follows:

$$\begin{aligned} LBP(x_c) = \sum _{i = 0}^{p-1} s(x_i - x_c)2^{p}, \end{aligned}$$

(5)

where p is the total number of the neighborhood pixels, $2^{p}$ is a binomial factor for each sign $s(x_i - x_c)$, and $s(\cdot )$ is a symbolic function:

$$\begin{aligned} s(x)=\left\{ \begin{aligned} 1,~&x \ge 0\\ 0,~&x<0. \end{aligned}\right. \end{aligned}$$

(6)

In the LBPCV model, intensity, gradient and LBP values of a pixel form a feature vector to fully describe a pixel. In the whole domain, three feature filter maps based on the original gray-scale image I can be obtained. Let x be a pixel in the derived vector-image U and $U_j$ be the jth filter map, and then define $P_{rj}^{x}$ calculating the histogram on image $U_j$ at pixel x in a patch with radius r, $P_{ij}$ and $P_{oj}$ represent the corresponding averages histograms of all pixels inside and outside the curve C, so the energy function for the LBPCV model is :

$$\begin{aligned} \begin{aligned} E_{LBPCV}(\bar{P}_i,\bar{P}_o,C)&= \int _{in(C)}\sum _{j=1}^J\lambda _{1j}W_1(P_{ij},P_{rj}^{x})dx\\&+\int _{out(C)}\sum _{j=1}^J\lambda _{2j} W_1(P_{oj},P_{rj}^{x})dx+ \mu \int _Cds, \end{aligned} \end{aligned}$$

(7)

where J is the total number of filter maps, here $J=3$, and $\mu$, $\lambda _{1j}$, $\lambda _{2j}$ are experiential positive values, $\bar{P}_i$ and $\bar{P}_o$ are average vector histograms which are equivalent to $(P_{i1},P_{i2},...,P_{iJ})$ and $(P_{o1},P_{o2},...,P_{oJ})$. The LBPCV model suggests that if various features of a image be in full use, they can complement each other in an integrated vector CV segment model and finally achieve a win-win effect as a whole.

3 Proposed Method

In this section, we will creatively put forward a local contrast pattern (LCP) texture descriptor, which contains two components: differential contrast $\xi$ and orientation $\theta$. Next we will establish a statistical histogram-based CV model with a truncated Gaussian kernel function on LCP feature maps to segment color texture region with the capacity of anti-jamming.

3.1 Local Contrast Pattern (LCP)

Also enlightened by the Weber Local Descriptor (WLD) in extracting gradient direction, we turn to adopting the texture direction estimation by generating the local horizontal and vertical gradient. Thus, in order to fully describing texture structure, we combines the two local component, differential contrast $\xi$ with orientation $\theta$, as a novel local operator named local contrast pattern (LCP). A LCP value of a current pixel $x_c$ is computed as Fig. 1 illustrated.

Fig. 1.

Illustration of the computation of the Local Contrast Pattern (LCP) descriptor.

Differential Contrast $\boldsymbol{\xi }$. The differential contrast $\xi$ of the LCP descriptor highlights the differential contrast between the center pixel and points around it as the intensity variation of the current pixel. In the same $3\times 3$ local window as LBP, $\xi$ of the pixel $x_c$ is computed as follows:

$$\begin{aligned} \xi (x_c) = \sum _{i=0}^{p-1}(x_i-x_c). \end{aligned}$$

(8)

Unlike the LBP operator, the differential contrast $\xi$ feature sums all the intensity differences $(x_i-x_c)$ instead of binarizing the comparison results. And it preserves the local intensity discrepancy so that it can reflect the variability of gray value effectively and further better discriminate different objects in segmentation application. To verify the performance of $\xi$, we test two images to compare both operators in Fig. 2. The original images are shown in (a), including synthetic and natural images, (b) and (c) are the normalized LBP maps and the normalized differential contrast maps of the LCP descriptor respectively. It is obvious that they both have the ability to describe texture, but LBP highlights the texture features of background simultaneously or even more prominent than that of the foreground, while $\xi$ in LCP suppresses some global clusters and distinguish different image features successfully. In addition, $\xi$ is more accurate than LBP in the extraction of texture details when referring to the head and legs of zebra image. Moreover, under the influence of light or shadow, there seems to be no serious disturbance in $\xi$ map.

Fig. 2.

Comparison of LBP and differential contrast $\xi$ of LCP. (a) original images, (b) normalized LBP maps, (c) normalized differential contrast maps in LCP.

Orientation $\boldsymbol{\theta }$. The orientation $\theta$ of LCP is computed based on the gradient orientation as described in [5]. Consider the larger gradient value corresponds to the bigger significant intensity variation, we utilize vertical and horizontal templates to filter image and then combine this two gradient components to calculate gradient angle. We also turn to the neighboring pixels centered in pixel $x_c$ in the $3\times 3$ window with the formula below:

$$\begin{aligned} \theta '(x_c) = arctan(\frac{\nu _s^0}{\nu _s^1}), \end{aligned}$$

(9)

where $\nu _s^0=x_7-x_3$, $\nu _s^1=x_5-x_1$, and $\theta '\in [-\pi /2,\pi /2]$. For unsigned data representation and convenient calculation, we perform a mapping $f:\theta '\xrightarrow {}\theta$. Thus, the final orientation map $\theta$ of the LCP operator is defined as follows:

$$\begin{aligned} \theta (x_c) = arctan2(\nu _s^0,\nu _s^1) + \pi \end{aligned}$$

(10)

with

$$\begin{aligned} arctan2(\nu _s^{0},\nu _s^{1})= {\left\{ \begin{array}{ll} \theta ', &{} \nu _s^{0}> 0,\text { and }\nu _s^{1}>0\\ \pi +\theta ', &{}\nu _s^{0}> 0,\text { and }\nu _s^{1}<0\\ \theta '-\pi , &{}\nu _s^{0}< 0,\text { and }\nu _s^{1}<0\\ \theta ', &{}\nu _s^{0}<0,\text { and }\nu _s^{1}>0 \end{array}\right. } \end{aligned}$$

where $\theta \in [0, 2\pi ]$. The final orientation $\theta$ considers not only the value of $\theta '$ computed in Eq. 9 but also the sign of $\nu _s^{0}$ and $\nu _s^{1}$.

3.2 Statistical Histogram Fitting Energy Based on LCP

Consider a given gray image I whose image domain $\varOmega$ can be separated into two regions $\varOmega _i$ and $\varOmega _o$ by a closed evolving curve C. $T_{\xi }$ and $T_{\theta }$ are two LCP feature maps. For better localization property, we introduce a region-scalable kernel function that make the histogram of the pixel closer to the center one weight higher and decrease quickly to zero when the distance is beyond the parameter scope. So this LCP driving fitting energy based on local histogram at center pixel x of feature maps can be defined by:

$$\begin{aligned} \begin{aligned} E_x^{LCP}({P_{ki}^x},{P_{ko}^x},C)&= \lambda _1\!\!\int _{in(C)}\!\!\sum _{k=1}^2\!\gamma _k K_{\sigma }\!(x-y)W_1(P_{ki}^x, P_{kr}^y)dy \\&+\!\lambda _2\!\!\int _{out(C)}\!\!\sum _{k=1}^2\!\gamma _k K_{\sigma }\!(x-y)W_1(P_{ko}^x, P_{kr}^y)dy, \end{aligned} \end{aligned}$$

(11)

where $\lambda _1$, $\lambda _2$ are the weighting parameters which control the image data driven force, $\gamma _k$ is a choosing parameter with $\gamma _k\in \{0,1\}$, $k=1$ represents the differential contrast map of the LCP descriptor and $k=2$ indicates the orientation map of the LCP descriptor, and if $\gamma _k=1$, the current kth map will be working in the energy fitting model. $W_1(\cdot )$ is the linear Wasserstein distance defined as Eq. 4, $K_{\sigma }(x-y)$ is the truncated Gaussian kernel. $P_{kr}^y$ is the statistical histogram calculated in the neighborhood centered in a pixel y with radius r based on the kth LCP map, $P_{ki}^x$ and $P_{ko}^x$ are the averages of the statistical histograms of the pixels inside and outside curve C in a local patch whose size is defined by the distance parameter $\rho$ of $K_{\sigma }(x-y)$ with a center x.

According to the level set methods, the active contour C is implicitly expressed as the zero level of a Lipschitz function $\phi :\varOmega \xrightarrow {}R$, so Eq. 11 can be rewritten as:

$$\begin{aligned} \begin{aligned} E_x^{LCP}({F_{ki}^x},{F_{ko}^x},\phi )&= \lambda _1\!\!\int _{\varOmega }\!\!H\!(\!\phi (x)\!)\!\!\sum _{k=1}^{2}\!\gamma _k K_{\sigma }\!(x-y)\\&\times \int _0^{L}\!\!|F_{ki}^x(z)\!-\!F_{kr}^y(z)|dzdy \!+\! \lambda _2\!\!\int _{\varOmega }\!\!(1\!-\!H\!(\!\phi (x)\!))\\&\!\!\times \!\!\sum _{k=1}^{2}\!\gamma _k K_{\sigma }\!(x-y)\!\!\int _0^{L}\!\!|F_{ko}^x(z)\!-\! F_{kr}^y(z)|dzdy, \end{aligned} \end{aligned}$$

(12)

where $H(\cdot )$ is the Heaviside function, and the cumulative distributions $F_{ki}^x(z)$ and $F_{ko}^x(z)$ are described as follows:

$$\begin{aligned} \left\{ \begin{aligned} F_{ki}^x(z)&=\frac{\int _\varOmega \!H\!(\!\phi (x)\!)K_\sigma \!(x-y)F_{kr}^y(z)dy}{\int _\varOmega \! H\!(\!\phi (x)\!)K_\sigma \!(x-y)dy} \\ F_{ko}^x(z)&= \frac{\int _\varOmega \!(1-H\!(\!\phi (x)\!))K_\sigma \!(x-y)F_{kr}^y(z)dy}{\int _\varOmega \! (1-H\!(\!\phi (x)\!))K_\sigma \!(x-y)dy}.\end{aligned}\right. \end{aligned}$$

(13)

For color texture image, we will embed color information into the local histograms based on the LCP texture feature maps. Given a 3-dimensional color image U for a two-phase segmentation. We first make LCP feature transform on single color component diagram $U_j$ and get feature maps $T_{\xi j}$ and $T_{\theta j}$, so that texture information is extracted on each channel and a vector-valued feature map is obtained. Then for every pixel point, an integrated local histogram and its cumulative histogram are computed in a block contending all color channels instead of calculating in a patch of an one-dimensional image:

$$\begin{aligned} P_r^x(\alpha )=\frac{\sum _{j = 1}^{d}|\{z\in N_r^x\cap \varOmega :T_{k j}(z)=\alpha \}|}{d\times |N_r^x\cap \varOmega |},d=3. \end{aligned}$$

(14)

$$\begin{aligned} F_r^x(\alpha )=\frac{\sum _{j = 1}^{d}|\{z\in N_r^x\cap \varOmega :T_{k j}(z)\le \alpha \}|}{d\times |N_r^x\cap \varOmega |},d=3. \end{aligned}$$

(15)

where $k=1$ denote $\xi$ and $k=2$ denote $\theta$.

Our goal is to minimize the energy for all pixels in the whole image domain. Thus, the total local histogram fitting energy based on LCP by integrating Eq. 12 in $\varOmega$ is expressed as below:

$$\begin{aligned} \begin{aligned} E_{LCP}({F_{ki}^x},{F_{ko}^x}, \phi )&=\lambda _1\!\int \!\!\!\int _{\varOmega }\!\!H\!(\!\phi (x)\!)\!\!\sum _{k=1}^{2}\!\gamma _kK_{\sigma }\!(x-y)\!\!\int _0^{L}\!\!|F_{ki}^x(z)\!-\! F_{kr}^y(z)|dzdydx\!\\&\!\!+\lambda _2\!\!\int \!\!\!\int _{\varOmega }\!\!(1\!-\!H\!(\!\phi (x)\!))\sum _{k=1}^{2}\gamma _kK_{\sigma }\!(x-y)\!\!\int _0^{L}\!\!|F_{ko}^x(z)\!-\! F_{kr}^y(z)|dzdydx. \end{aligned} \end{aligned}$$

(16)

Equation 16 is the objective function to be minimized, it is called a data fitting energy and plays a key role in driving the deforming contour C to evolute.

4 Experimental Results

In this section, we tested a number of color texture images to verify the effectiveness as well as the robustness for illumination, noise and curve initialization of the proposed model. In order to directly proof the LCP descriptor playing a key role in texture discrimination, we have done an additional set of experiments without a kernel function in our novel method. Besides, three unsupervised color-texture partitioning methods including a classical strategy called JSEG method [21], an approach by aggregating super pixels (SAS) [23] and an segmentation based on multiscale quaternion Gabor filters and splitting strategy (MQGF) [9] were used for color-texture segmentation comparison.

Color Texture Images. Figure 3 compared the results on three color texture images from “Berkeley” database listed in (a) whose target objects were obvious but background interferences were multiply. (b) and (c) were the results of our modified scheme with and without incorporating the kernel function $K_\sigma$. As we expect, the integrated histograms calculating in a vector feature image block were also valid as in single-image cases. The salient regions with inhomogeneous color and texture were captured more successfully when $K_\sigma$ enhanced the effect. (d)-(f) depicted the results of three comparisons: the classical JSEG method, and the latest SAS and MQGF method. Disappointed, many unwanted contours were generated and texture details were neglected. Clearly, our model was also superior in terms of both precision and accuracy.

Fig. 3.

Segmentation results on natural color images. (a) original images, (b) results of our method using $\xi$, (c) results of our method using a combination of $\xi$ and $K_\sigma$, (d) results of the JSEG method, (e) results of the SAS method, (f) results of the MQGF method. (Color figure online)

Robustness for Illumination and Noise. In order to verify the robustness for illumination and noise still worked in color texture segmentation versions, we made some additional contrast tests in more extreme circumstances. We took three color texture images with special processing: the first one was set with a light source at the top of the object, the second one was added both uneven illumination and Gaussian white noise of variance 0.02, and the third one was polluted by a combination of Gaussian white noise of variance 0.02 and salt-and-pepper noise of density 0.05. (b) were the results of our method using $\xi$ while (c) were the results of our method using a combination of $\xi$ and $K_\sigma$. In spite of bad environment, the results of our strategy shown in (b) and (c) were unaffected completely, and they maintained accurate segmentations as always. (d)–(f) were the results of three contrasting approaches. Conclusion got from the distribution of cutting curves was that the bad influence of illumination to all the comparisons was very deadly, and their resistance to noise was also weaker than ours. In this experiment, we had increased the difficulty specifically, but it turns out that the robustness was still particularly evident in our novel model (Fig. 4).

Fig. 4.

Experimental results for illumination and noise. (a) original images, (b) results of our method using $\xi$, (c) results of our method using a combination of $\xi$ and $K_\sigma$, (d) results of the LH model, (e) results of the LBPCV model, (f) results of the LGDF model.

5 Conclusion

In this paper, the local contrast pattern(LCP) descriptor is proposed to tackle intensity inhomogeneity and orientation information in texture image segmentation. We first use it to make image feature conversion and then we incorporate the feature maps into a histogram-based CV model with a Gaussian kernel function. The experimental results demonstrate that the proposed method can segment both gray-scale texture image and color-texture image. Furthermore, our method is proved to be very robust for image quality and curve initialization.