Spatially Varying Weighting Function-Based Global and Local Statistical Active Contours. Application to X-Ray Images

Abstract

Image segmentation is a crucial task in the image processing field. This paper presents a new region-based active contour which handles global information as well as local one, both based on the pixels intensities. The trade-off between these information is achieved by a spatially varying function computed for each contour node location. The application preliminary results of this method on computed tomography and X-ray images show outstanding and efficient object extraction.

1 Introduction

Image segmentation is a fundamental problem in the fields of computer vision, image processing and pattern recognition. It is the crucial task aiming at separating the domain of interest from the background to change the representation of the image into something that is more meaningful and easier to analyze by the next higher analysis stages. Hence, its outcomes have indisputably a direct effect on the analysis issue. For delicate applications such as processing radiographic images in both fields of industry and medicine, an accurate segmentation is more than ever required, since a bad interpretation or a false diagnosis lead, sometimes, to irreparable harm to the human patient and the industrial plant in question [1]. In the applications which involve X-ray images, the distinction of the region of interest structure is complicated by inherent noise, artifacts and intensity inhomogeneity. Image segmentations in such cases is not easy for numerous reasons: Firstly, partitioning the image into non overlapping regions and extracting regions of interest require a tradeoff between the computational efficiency of the involved algorithm, its degree of automation and the accuracy of its outcomes. Secondly, image noise, intensity inhomogeneity linked to the image acquisition, and poor contrast are very difficult to reckon with in segmentation algorithms without the user interacting [2]. Due to the causes previously evoked, designing a robust and efficient segmentation method is still a difficult problem in practical applications. To overcome these difficulties, segmentation with deformable models or active contours seems to be quite suitable to extract complex structures with complex texture, inhomogeneity and complex shapes. The most important reason for this is the fact that the active contours can incorporate global and local view of image segmentation by assessing continuity and curvature combined with the local edge strength and/or the region information [3]. Furthermore, by considering the boundary as a whole, a structure is imposed to the solution. As a result, the overall structure shape to be extracted is recovered by means of one smooth curve located as close as possible to the real boundary. Moreover, such boundary description can be readily used by subsequent applications [3]. Since they were proposed, active contours have gained researchers interest and have been widely applied in the computer vision field with promising results. The central idea of active contour models is to evolve a curve under some constraints from a given image to detect the desired objects. Based on the nature of constraints and image features, the existing active contour models can be roughly categorized into two classes: edge-driven and region-driven. Edge-driven are called so because the information used to drive the curves to the edges is strictly along the boundaries. Edge-driven models show satisfactory results when segmenting images with distinct edges. Nevertheless, these models are sensitive to noise and initial conditions and sometimes are with severe boundary leakage problems. With the region-driven ones [4], the inner and the outer region defined by the model are considered, making this model class well-adapted to situations for which it is difficult to extract boundaries from the target and is less sensitive to noise and to the initial position of the curve.

The present paper deals with object extraction from images by means of a region-based active contour that uses global and local statistical pressures forces controlled by an adaptive weighting function. The remainder of the paper is organized as follows: In Sect. 2, we introduce the mathematical foundation of active contours. In Sect. 3, we present our method for object extraction. Section 4 is dedicated to experimental results. We draw the main conclusions in Sect. 5.

2 Background

2.1 Active Contours

The active contour models for image segmentation, known as snakes, are characterized by a curve $\mathbf c (s)= [x(s), y(s)]', s =[0 \ 1]$ which evolves towards certain image features under forces to minimize the energy [5]

$$\begin{aligned} E(\mathbf c )=\int _0^1( E^{int}(\mathbf c (s))+E^{ext}(\mathbf c (s)))ds \end{aligned}$$

(1)

where s is the curvilinear abscissa, $E^{int}$ the internal energy of the contour which maintains a certain degree of smoothness and controls the snake nodes spacing along the contour and $E^{ext}$ the image energy responsible for driving the contour toward edges and computed from the image data. The minimization of Eq. (1) leads to an iterative solution that governs the model evolution as given in [5].

$$\begin{aligned} \left\{ \begin{array}{l} x_t=(A+\gamma I_d)^{-1}(\gamma x_{t-1}+\nabla E^{ext}_x(x_{t-1},y_{t-1}))\\ y_t=(A+\gamma I_d)^{-1}(\gamma y_{t-1}+\nabla E^{ext}_y(x_{t-1},y_{t-1})) \end{array} \right. \end{aligned}$$

(2)

If the model is made of N nodes, then, A is a $N*N$ matrix, $I_d$ an identity matrix sized as A, $\gamma$ an evolution coefficient, $x_t$ and $y_t$ are the model nodes coordinates at the iteration t. $\nabla E^{ext}_x(x_t,y_t)$ and $\nabla E^{ext}_y(x_t,y_t)$ are the external forces of the input image at the model nodes locations in the x and y direction respectively.

2.2 External Forces

In addition to the standard external force proposed in [5], a variety of external forces have been proposed to improve the performance of snakes. The external forces can be generally classified as dynamic forces and static forces [6]. The dynamic forces are those that depend on the snake and, as a result, change as the snake deforms. In turn, the static forces are those that are calculated from the image only, and remain unchanged as the snake deforms [6]. The pressure force given in Eq. (3) also called the balloon force [7] which is a useful dynamic force, is the inflation/deflation force that pushes the curve outward or inward. By introducing a pressure weight k for individual nodes as in [2, 4, 8–10], the model, regarding to the initialization issue, is strengthened since it can inflate and deflate independently and at the same time. The pressure force may be released to various forms. Indeed, diverse region information can be used to modulate the pressure weight and sign, so that the contour part shrinks when placed outside the object of interest and expands when placed inside.

$$\begin{aligned} F_B(\mathbf c (s^*))=k.{\varvec{n}}(\mathbf c (s^*)) \end{aligned}$$

(3)

${\varvec{n}}(\mathbf c (s^*))$ is the normal unit vector to the curve at $\mathbf c (s^*)$ and k the pressure weight. An example of this individual pressure weight is proposed in [11], where k was introduced as

$$\begin{aligned} k=p(z_{s^*}/O)-p(z_{s^*}/B) \end{aligned}$$

(4)

where $z_{s^*}$ is the gray value of the pixel falling on the snake node $\mathbf c (s^*)$, whereas $p(z_{s^*}/O)$ and $p(z_{s^*}/B)$ are, respectively, the conditional probability density functions of the object (O) and the background (B). The model does not make assumptions on the probability density function (pdf) of the object features. The problem is reduced, then, to an accurate estimation of the pdf of both object and background pixels gray-values, particularly in the case of non-simple gray-levels distributions [11].

3 The Proposed Model

Region-driven active contours can get more robust and global segmentation results when using global information, but mostly the object is distinguished by local variations. Thus, a compromise between global and local features is needed [12]. Furthermore, in image segmentation, both of object boundaries and fine structures can be detected accurately by incorporating the local neighbor region information. Meanwhile, to lessen the impact of noise and the possibility of getting stuck in local minima, the global region information plays an important role and particularly decreases the sensitivity to the contour initialization [10]. In this work, the averaged shifted histogram (ASH) method is used to model the gray levels pdf inside and outside the model curve and is exploited as global region information. Moreover, the local information is also taken into consideration to refine detection accuracy, however in a novel way. In fact, we combine, here, the advantages of global and local information where the contour evolution is held by these two kinds of information. On one hand, the global intensity allows the model expansion while incorporating global image information which improves the model robustness against noise and initial position. On the other hand, the local intensity information which is dominant near boundaries because of its sensitivity to the region transition, is integrated to operate near the object boundaries and then attract the model to them.

3.1 Progressing the Model Curve

To exploit the Bayesian weight of Eq. (4), one should have a good estimation of the pdf of the image data model. The simplest way to estimate the pdf of the pixels gray levels, is to compute the intensity histogram of the image, which is the tonal distribution in a digital image. Indeed, it shows the appearance frequency of gray levels and when normalized, it can roughly assess the density function of the gray-values. In fact, the histogram is a classical nonparametric density estimator probably dating back to the seventeenth century [13]. Nowadays, histogram remains an important statistical tool for summarizing data. Computing an image histogram is a trivial operation, but results for noisy images could be unusable for density estimation without some processing. Smooth versions of histogram have been proposed in the literature. Among them, we can cite the averaged shifted histogram ASH, which has been employed in the scope of pdf estimation to drive the active contour to the boundaries in [14]. Indeed, ASH has desirable properties and numerous advantages as we will see later.

Averaged Shifted Histogram ASH. The averaged shifted histogram [15] is a nonparametric probability density estimator derived from a collection of histograms. The ASH enjoys several advantages compared to a single histogram and a more complex non-parametric histogram-based pdf estimators like kernels estimators. As advantages, we can cite better visual interpretation, better approximation, and nearly the same computational efficiency in regards to the former [15, 16] and computation speed and the same efficiency compared to the latter. Indeed, the ASH provides a bridge between the histogram and advanced kernel methods, which are more computationally intensive. ASH method combines a set of m histograms generated with a certain shift into the bin borders [16]. It can be shown that when the number of histograms tends to infinity, the ASH approximates the kernel estimator [15]. In practice, ASH is implemented by generating an histogram with smaller bin width, and computing the discrete convolution with a triangular window. If $X_1,\ldots , X_n$ are samples of random variables with density f, the pdf estimation, with m histograms, of all samples X being in the finer interval $B_k$, using the ASH gives

$$\begin{aligned} \hat{f}(X)_{ASH}=\frac{1}{n.h}\sum _{i=1-m}^{m-1}\left( 1-\frac{|i|}{m}\right) v_{k+i} \qquad \forall x\in B_k \end{aligned}$$

(5)

where h, n and $v_k$ are, respectively, the bin width of the averaged histograms, the total number of samples and the number of observations falling in the sub-bin $B_k$. ASH strongly depends on the band width h. If we suppose that the underlying density is Gaussian, then, it can be shown that an optimal choice for h is [13]

$$\begin{aligned} h_{opt}=3.5\hat{\sigma }.n^{-1/3} \end{aligned}$$

(6)

where $\hat{\sigma }$ is a standard deviation estimate.

Therefore, the global force is a pressure one, whose weight is called here $k_{Global}$ and computed using Eq. (4) where the pdf is determined with the ASH method.

Fig. 1.

Local window

Deviation from the Central Pixel Gray-Level. By adding another pressure force to make the contour model progress, also, regarding to the local properties of the image, the contour can detect significant changes in the local neighborhood of the overall model nodes, like region transitions. Let call $k_{Local}$ this new pressure force weight which is computed as the central pixel gray level deviation from the pixels gray levels within a circular window centered at the current model node. As shown in Fig. 1, this window and the model curve, split jointly, the local neighborhood of the model node into local interior and local exterior.

$k_{Local}$ is the deviation from the central pixel gray level and is computed as

$$\begin{aligned} k_{Local}=\sqrt{\sum _{i=1}^{N^{in}}\tau (c(s^*)),x^{in}_{i})(z_{s^*}-z_{x^{in}_{i}})^2}-\sqrt{\sum _{j=1}^{N^{out}}\tau (c(s^*),x^{out}_{j})(z_{s^*}-z_{x^{out}_{j}})^2} \end{aligned}$$

(7)

where $\mathbf c (s^*)$ is the center of the circular neighborhood, $z_{x^{in}_{i}}$ the gray level of the $i^{th}$ pixel $x^{in}_{i}$ in the interior local neighborhood, $z_{x^{out}_{j}}$ the gray level of the $j^{th}$ pixel $x^{out}_{j}$ in the exterior local neighborhood. $N^{in}$ ($N^{out}$) is the pixels number in the local interior (local exterior). We call $\tau$ a weighting function related to the distance between the central pixel and the other ones in the circular window, so that the sum of the weights in the interior/exterior local neighborhood equals to one ‘1’. $\tau$ can be chosen as a gaussian kernel function as done in [3], for example, to handel inhomogeneities. The values of $k_{Local}$ are normalized so that they belong to $[-1 \ 1]$. When a snake node is in a homogeneous local region, this weight moves towards zero since the intensities are quite similar and then global information only controls the model progression for this node. However, when the node is near a region boundary as in Fig. 1 where the local interior is homogeneous and the local exterior is not, the first term of Eq. (7) tends to zero whereas the second term does not. This produces a negative weight and the model will expand at this node location (positive normal vectors point inward by convention here). The iterative evolution equation of the model is given by

$$\begin{aligned} \left\{ \begin{array}{l} x_t=(A+\gamma I_d)^{-1}(\gamma x_{t-1}+ [\omega k_{Local} +(1-\omega )k_{Global}]N_x)\\ y_t=(A+\gamma I_d)^{-1}(\gamma y_{t-1}+ [\omega k_{Local} +(1-\omega )k_{Global}]N_y) \end{array} \right. \end{aligned}$$

(8)

where $N_x$ and $N_y$ are, respectively, the normal unit vector components in the x and the y direction and $\omega$ the spatially adaptive trade-off function between the local and global force.

The Spatially Varying Function ${\varvec{\omega }}$. The global and the local pressures forces, in this work, are encouraged to be complementary rather than competitive. So, the $\omega$ function is designed to remove the two forces competition. When the model is far away from object boundaries, the global forces should have the decisive effect on the contour evolution. Furthermore, near the boundaries, the evolution should be taken over by the local forces. The choice of $\omega$ is done by the following observation: when a local neighborhood of a node is made up of a homogeneous part or a part with a slightly variable intensity between the local interior and the local exterior, then, this neighborhood is very likely within the object or within the background. Moreover, when the intensity vary too much between the mentioned local neighborhoods, then the latter is probably in a transition region. This can be supported by Fig. 2. If $\mu ^{in}_{loc}$ and $\mu ^{out}_{loc}$ ($\sigma ^{in}_{loc}$,$\sigma ^{out}_{loc}$) are the mean (the standard deviation) of the local interior and the local exterior, respectively, then, $\omega$ is empirically computed for each node as

$$\begin{aligned} {\begin{matrix} \omega =1 \ \text {if}\ |\mu ^{in}_{loc}-\mu ^{out}_{loc}|>\sigma ^{in}_{loc}+\sigma ^{out}_{loc}, \qquad \omega =0 \ \text {otherwise} \end{matrix}} \end{aligned}$$

(9)

Fig. 2.

Local neighborhoods centered at different nodes of the snake model

4 Experiments

In the proposed active contour model, the pdf estimation is of a major importance. Usually, kernels-based estimators are employed in the scope of deformable models evolution using a non-parametric pdf estimation. As the global statistical pressure forces computed by the mean of pdf estimation, have in charge the model expansion, we begin our experiments by showing an example of an image histogram and estimated pdfs using the ASH and the kernels-based (Parzen window) methods. Figure 3 shows an X-ray image of a crater crack, while are depicted in Fig. 4 its histogram, its ASH and kernels-based pdfs estimations. We can note that the various histogram modes, for the ASH-based estimation, are quite visible, whereas they are over smoothed for the kernels-based one. However, to make the latter better in terms of histogram fitting, the kernel bandwidth h [17] should be refined accordingly, instead of computing it just by the rule of thumb [18]. Consequently, a supplementary running-time will be required.

Fig. 3.

X-ray image of a crater crack

Fig. 4.

Histogram, ASH and Kernels-based pdf estimations

Fig. 5.

ASH (left) and kernels-based (right) active contours results beginning from the same initialization

We continue these tests by using global pressure forces alone to drive the active contour model in a synthetic image, illustrated in Fig. 5, to show the results of the ASH and kernels-based active contours in terms of object segmentation and progression time. Since the segmentation accuracy, in the presented method, is achieved by the local forces, these preliminary results ascertain the opportunity to use ASH-based pdf estimation in order to speed up the contour progression in comparison to kernels-based pdf estimation. Indeed, the running time takes 3.5 s for the ASH-based active contour instead of 9 s for the kernels-based one. This shows that the ASH method could be a good alternative for histogram-based pdf estimation for active contours instead of more sophisticated and time-consuming methods.

In the next experiments, we show the active contour progression with only the local forces. Far from the boundaries, the local forces based model does not progress, since local forces can operate, as explained before, exclusively at the model nodes neighborhood. Whereas, when placed near the boundaries, these forces could push the model curve to them as shown by the Fig. 6. For this example, the neighborhood is chosen to have a radius of 3.

Fig. 6.

Left: Initialization near the boundaries (solid line) and the local forces-based model final contour (in dashed line). Right: Initialization far from the boundaries (solid line) and the local forces-based model final contour (in dashed line).

The following tests are carried out on real X-ray-based images. The image shown in Fig. 7 represents a region of interest (ROI) of X-ray image of a welded joint subjected to segmentation. In this case, the task is to extract the weld defect indications from the ROI. In this example, the opportunity of using local forces is highlighted. Indeed, when the model is faced to blurred edges, local forces bring more precision to the weld defect extraction as shown by this figure. Furthermore, leakage problems could happen when edges are weak or in presence of inhomogeneities when using only global information, as shown in Fig. 8, which represents a brain computed tomography (CT) image. In this case, the challenge is to exact, as accurately as possible, the shape of the lesion from healthy tissues, for surgery purpose, for example, which is successfully done with our model.

Fig. 7.

From Left to right: Initialization, final contour without local forces, and final contour of the proposed model.

Fig. 8.

Contour initialization in the top. Down:(left) Final contour without local forces with a leakage problem, (right) Final contour of the proposed model.

Fig. 9.

Final contours of the proposed model on two radiographic images.

To finish this section, we give some other results of applying the proposed active contour on weld X-ray and CT images shown in Figs. 9 and 10. It is to note that the local window radii for X-ray images and for CT images have been empirically chosen equal to 5 and 20, respectively.

Fig. 10.

Final contour on a computed tomography brain image.

5 Conclusion

In the present paper, we have proposed a method to extract objects from X-ray images based on local and global information and a spatially varying trade-off function that removes the competition between them. Results seem to be very promising since the model avoids leakage problem, when the boundaries are weak, and performs better segmentation than a model based on global information used alone.