Edge Aware Geometric Filter for Ultrasound Image Enhancement

Abstract

Despeckling of ultrasound images is essential for subsequent computational analysis. In this paper, an edge aware geometric filter (GF) is proposed for speckle reduction. The behaviour of conventional GF is approximated using commonly used functions like unit step. These approximations help in identifying the natural relationship between GF and other existing spatially adaptive filters. Subsequently, the modifications in GF framework are proposed to take the advantage of edge characteristics. The proposed filter requires almost no parameter tuning and provides good quality outputs for synthetic as well as real ultrasound images. It is compared with the state-of-the-art speckle reducing filters. Improvements of 10.46% and 42% are noticed in mean square error and figure of merit, respectively.

1 Introduction

Ultrasound (US) images contain granular patterns generated from the constructive and destructive interferences of the backscattered US pulse echoes. These patterns are collectively known as speckle [1, 2, 6, 11, 17]. Despeckling of US images depends on subsequent applications. Some applications like tissue characterization, require the speckle to be selectively removed only from the regions of low clinical significance like blood [18]. However, this paper focuses on enhancing the US images to make them suitable for applications involving automatic computational analysis like segmentation. Therefore, the speckle is removed from the entire image, irrespective of the underlying region.

Several speckle reducing filters are reported in literature for US image enhancement [5, 12]. Spatially adaptive filters, for example Lee filter [16] and Kuan filter [15], are some of the well known traditional filters. More advanced filters are based on anisotropic diffusion [3, 19,20,21]. These filters prohibit filtering across the edges to preserve object boundaries. Apart form these, wavelet transform based filters [22] and non-local mean (NLM) filters [10] are also popular. In contrast to the local averaging filters [4], the NLM filters reduce speckle by using a weighted average of non-local image regions. Although, all these filters provide good outputs, their performance is highly sensitive to the tuning of several implementation parameters. It increases the complexity and also leads to catastrophic results in case of inefficient parameter tuning.

In this paper a despeckling filter is proposed which has its roots in the old concepts of geometric filtering [7]. Geometric filter (GF) [7] is a well known speckle reducing filter. Although it aggressively suppresses speckle to quickly reach a stable output, the amount of speckle reduction is not as good as the more advanced diffusion based filters [11]. The key difference is that the GF is unable to use the edge characteristics of the image. In this paper the behaviour of GF is approximated using the function commonly used for signal analysis, the unit step and signum function. This helps us in identification of the natural relationship between GF and spatially adaptive filters like Lee filter. Which in turn enables us to take the advantage of the edge characteristics in GF framework. The proposed edge aware GF requires almost no parameter tuning. It is compared with the existing state-of-the-art filters and a competitive performance is observed.

The paper is organized as follows. Section 2 presents the approximation of the behaviour of GF and its relationship with the spatially adaptive filters. The section also contains details about the proposed filer. Section 3 includes experimental results which are compared with the existing filters. Finally conclusions are presented in Sect. 4.

2 Proposed Filter

This section provides details about the relationship between GF and other existing spatially adaptive filters. Later in the section the edge aware GF is proposed which eliminates the limitations of the conventional GF to provide efficient speckle reduction.

2.1 Geometric Filter (GF)

The GF considers lines of pixels in all four directions, East to West, North to South, North-east to South-west and North-west to South-east [8, 9]. These lines which represent gray level profiles, are used to create binary images. The complementary convex hull algorithm is applied on these binary images to make the gray level profile smooth. Accordingly, each pixel in the image is filtered in all four directions to increase the homogeneity among the neighbourhoods. Filtering operation of GF can be easily described using certain rules. Let us assume $x\in \mathbb {R}^2$ represents a pixel and $x_1$ and $x_2$ are two neighbours of x lying opposite to each other. Further, $f_x$ is the intensity value of pixel x in the image $f:\varOmega \rightarrow \mathbb {R}_+, \varOmega \subset \mathbb {R}^2$. The intensity value is adjusted based on following rules:

$$\begin{aligned} \text {if}&\quad f_{x_1}\ge f_x+2&\text {then} \quad f_x = f_x+1\end{aligned}$$

(1a)

$$\begin{aligned} \text {if}&\quad f_{x_1} > f_x\quad \text {and}\quad f_x\le f_{x_2}&\text {then} \quad f_x = f_x+1 \end{aligned}$$

(1b)

$$\begin{aligned} \text {if}&\quad f_{x_2} > f_x\quad \text {and}\quad f_x\le f_{x_1}&\text {then} \quad f_x = f_x+1\end{aligned}$$

(1c)

$$\begin{aligned} \text {if}&\quad f_{x_2}\ge f_x+2&\text {then} \quad f_x = f_x+1 \end{aligned}$$

(1d)

$$\begin{aligned} \text {if}&\quad f_{x_1} \le f_x-2&\text {then} \quad f_x = f_x-1 \end{aligned}$$

(1e)

$$\begin{aligned} \text {if}&\quad f_{x_1} < f_x\quad \text {and}\quad f_x\ge f_{x_2}&\text {then} \quad f_x = f_x-1\end{aligned}$$

(1f)

$$\begin{aligned} \text {if}&\quad f_{x_2} < f_x\quad \text {and}\quad f_x\ge f_{x_1}&\text {then} \quad f_x = f_x-1 \end{aligned}$$

(1g)

$$\begin{aligned} \text {if}&\quad f_{x_2}\le f_x-2&\text {then} \quad f_x = f_x-1 \end{aligned}$$

(1h)

The filtered image is obtained by iterative application of GF. The time complexity of GF is very low however the filter has some limitations. It works on the assumption that the noise appears as peaks and valleys of the same width in all regions. Considering the spatially correlated nature of speckle, this assumption is a critical assumption which limits the performance of the filter. Further, the GF fails to take the advantage of edge characteristics. Apart from inefficient filtering, it sometimes also results in tempered object boundaries.

2.2 A Different Look to Geometric Filter (GF)

The rules governing the mechanism of GF can be converted into mathematical expression using the standard unit step (u(.)) and signum (sgn(.)) functions. u(.) can be used to express the rule (1a) as:

$$\begin{aligned} f_x = f_x + u\left( f_{x_1} - f_x -2\right) \end{aligned}$$

(2)

Similar equation can be written for rule (1e) to obtain a combined expression for both the rules, (1a) and (1e), as:

$$\begin{aligned} f_x = f_x + u\left( f_{x_1} - f_x -2\right) -u\left( f_x - f_{x_1} -2\right) \end{aligned}$$

(3)

As we know, $sgn(k) = u(k) - u(-k)$, (3) can be rewritten as:

$$\begin{aligned} f_x = f_x + sgn\left( f_{x_1} - f_x\right) \end{aligned}$$

(4)

where the factor of 2 from the last two terms of (3) is ignored to get a concise expression. Ignoring the factor would require to use stopping criteria like insignificant intensity changes in successive iterations. Similar to (4), an expression for the rules (1d) and (1h) can be written as:

$$\begin{aligned} f_x = f_x + sgn\left( f_{x_2} - f_x\right) \end{aligned}$$

(5)

Further the rules (1b) and (1f) can be approximated as:

$$\begin{aligned} f_x = f_x + \frac{1}{2}\left( sgn\left( f_{x_1} - f_x\right) + sgn\left( f_{x_2} - f_x\right) \right) \end{aligned}$$

(6)

and similarly the rules (1c) and (1g). Combining these, all the rules can be represented using a single expression as:

$$\begin{aligned} f_x = f_x + 2\times sgn\left( f_{x_1} - f_x\right) + 2\times sgn\left( f_{x_2} - f_x\right) \end{aligned}$$

(7)

Similar expressions can be obtained for all four directions comprising all the eight neighbours, which would lead us to the following consolidated expression of the filter:

$$\begin{aligned} f_x^{t+1} = f_x^t + \sum _{x_1\in \chi }m\times sgn\left( f_{x_1}^t - f_x^t\right) \end{aligned}$$

(8)

where $\chi$ represent the neighbours of x and $m = 2$ is a constant. The superscript t denotes the time or iteration. Equation (8) is an approximated version of GF which is used to establish its relationship with other filters.

2.3 Relationship Between GF and Lee Filter

Mathematical expression of the Lee filter is given as:

$$\begin{aligned} f_x = \mu _g + k_x\times \left( g_x - \mu _g\right) \end{aligned}$$

(9)

where g and f represent the noisy and filtered images, respectively. $\mu _g$ is the local mean and k is a weight factor calculated using local and noise statistics. Equation (9) can be rewritten as:

$$\begin{aligned} f_x = g_x + \left( 1-k_x\right) \left( \mu _g - g_x\right) \end{aligned}$$

(10)

As we know, $\mu _g = \frac{1}{n(\chi )}\sum _{x_1\in \chi }g_{x_1}$, where $n(\chi )$ is the cardinality of neighbourhood. Thus, (10) can be rewritten as:

$$\begin{aligned} f_x = g_x + \frac{\left( 1-k_x\right) }{n(\chi )}\sum _{x_1\in \chi }\left( g_{x_1} - g_x\right) \end{aligned}$$

(11)

We can replace g with $f^t$ to reflect iterative filtering and also rewrite (11) as:

$$\begin{aligned} f_x^{t+1} = f_x^t + \frac{\left( 1-k_x\right) }{n(\chi )}\sum _{x_1\in \chi }\vert f_{x_1}^t - f_x^t\vert . sgn\left( f_{x_1}^t - f_x^t\right) \end{aligned}$$

(12)

where gives the magnitude. If we convert m from (8) into a variable and factorized it into two factors $m_1$ and $m_2$ as:

$$\begin{aligned} m_1 = \frac{\left( 1-k_x\right) }{n(\chi )}; \quad m_2 = \vert f_{x_1}^t - f_x^t\vert \end{aligned}$$

(13)

we can see that (12) boils down to (8). This establishes the relationship between GF and Lee filter. Also this shows that the behaviour of GF can be easily adjusted with the careful selection of $m_1$ and $m_2$.

2.4 Proposed Filter

Although GF has a notion of direction, it is unable to differentiate between an edge pixel (a pixel lying on an edge) and other pixels. It gives equal weightage to all the neighbouring pixels during filtering. However, for an edge pixel, the neighbours lying in the direction of edge always have a natural dependency. Therefore, these neighbours should have higher weightage during the intensity adjustments. This is the key idea of the proposed filter which helps in increasing the homogeneity along the direction of edges and in turn reducing the speckle. The simplest way to implement the proposed idea is to define $m_2$ as:

$$\begin{aligned} m_2 = |\pi _{\overrightarrow{xx_1}}\left( \overrightarrow{\rho }\right) | \end{aligned}$$

(14)

where $\overrightarrow{\rho }$ is a unit vector in the direction of the edge passing through the pixel x. The $\pi _{\overrightarrow{xx_1}}(\overrightarrow{\rho })$ gives scalar projection of $\overrightarrow{\rho }$ onto the unit vector $\overrightarrow{xx_1}$. The $m_2 \in \left[ 0,1\right]$ attains the maximum value when the dependency of $\overrightarrow{xx_1}$ on the edge is maximum. $\overrightarrow{\rho }$ can be obtained using any edge indicator. In this work gradients are used. Let us say i and j represent the co-ordinate axis of the image f, then the gradient $\nabla f$ is given as:

$$\begin{aligned} \nabla f = (\partial f_i, \partial f_j) \end{aligned}$$

(15)

where $\partial f_i$ and $\partial f_j$ are partial derivatives of f in the directions i and j, respectively. The $\overrightarrow{\rho }$ can be defined using $\nabla f$ as:

$$\begin{aligned} \overrightarrow{\rho }= \frac{\overline{\nabla f}}{\vert \overline{\nabla f}\vert } \end{aligned}$$

(16)

where $\overline{\nabla f} = (-\partial f_j, \partial f_i)$ gives a vector parallel to the edge. Now, with the definition of $m_2$ in (14), we can take $m_1 = 1$ without the loss of generality. Accordingly the expression of the proposed filter is given as:

$$\begin{aligned} f_x^{t+1} = f_x^t + \sum _{x_1\in \chi }\bigg |\pi _{\overrightarrow{xx_1}}\left( \frac{\overline{\nabla f^t}}{\vert \overline{\nabla f^t}\vert }\right) \bigg |\times sgn\left( f_{x_1}^t - f_x^t\right) \end{aligned}$$

(17)

The proposed definition of $m_2$ helps in filtering the image according to the edge characteristics. However, our objective is not just to remove speckle but also to enhance the object boundaries. Therefore, a simple modification in the definition of $m_2$ is done as: $m_2 = |\pi _{\overrightarrow{xx_1}}\left( \overrightarrow{\rho }\right) |-0.5$. This negative offset of 0.5 helps in increasing the difference between the pixels belonging to different regions and in turn enhances the object boundaries.

3 Experimental Results and Discussion

The proposed filter is tested on synthetic and real US images. The filtering performance is compared with Lee [16], GF [7], DPAD (detail preserving anisotropic diffusion) [3], OBNLM (optimized Bayesian non-local mean filter) [10] and SBF (squeeze box filter) [23]. The structural similarity measure index (SSIM) [24], figure of merit (FoM) [20] and commonly used mean square error (MSE) are used as evaluation metrics. The mathematical expressions of the evaluation metrics are as follows:

SSIM: it is a measure of structural similarity between the noisy and filtered images:

$$\begin{aligned} \text {SSIM} = \frac{\left( 2\mu _f\mu _{\hat{f}}+c_1\right) \left( 2\sigma _{f}\sigma _{\hat{f}}+c_2\right) }{\left( \mu _f^2+\mu _{\hat{f}}^2+c_1\right) \left( \sigma _{f}^2+\sigma _{\hat{f}}^2 +c_2\right) } \end{aligned}$$

(18)

where f is the filtered image and $\hat{f}$ is the noise free reference image. $\mu _f$ and $\sigma ^2_f$ are mean and variances of the image f. $c_1$ and $c_2$ are two small constants added to provide stability. The value of SSIM varies between 0 and 1, where 1 represents identical images in terms of structural similarity.

FoM: it is also known as Pratt’s figure of merit and calculated as:

$$\begin{aligned} \text {FoM}=\frac{1}{max\left\{ {N}, N_{ideal}\right\} }\sum _{i=1}^{{N}}\frac{1}{1+d_i\alpha } \end{aligned}$$

(19)

where N and $N_{ideal}$ represent the number of edge pixels obtained from filtered and reference images using the Canny edge detector. The $d_i$ is the Euclidean distance between an edge pixel obtained from filtered image and nearest edge pixel from reference image, and $\alpha$ is a constant set as 1/9. The value of FoM also varies between 0 and 1, where 1 represents the perfect edge preservation.

MSE: it is the measure of intensity differences between the filtered and reference images.

$$\begin{aligned} \text {MSE}=\frac{1}{N\times N}\sum _{x=1}^{N\times N}\left( \hat{f_x}-f_x\right) ^2 \end{aligned}$$

(20)

where $N\times N$ represent the total number of pixels in the considered image. Low values of MSE are desirable.

The implementation parameters of different filters require tuning to result in the best values of the evaluation metrics. For parameter tuning, initially any one parameter, like the number of iterations (itr), is fixed and rest of the parameters for example step size (dt) for DPAD, smoothing parameter (h) for OBNLM and localization parameter ($\sigma _g$) for SBF are varied over a very fine grid. The parameters associated with image regions for example window size (W), patch size $(\alpha )$ and search area (M) are varied over an integer grid. Later, the previously fixed parameter is varied to get the complete set of optimized parameters. In contrast to the existing filters, the proposed filter requires only the tuning of itr, which can also be eliminated by using any stopping criteria.

Fig. 1.

Synthesized phantom and simulated noisy image.

Fig. 2.

Filtered outputs of phantom image obtained using (a)–(h) Lee, GF, DPAD, OBNLM, SBF, and the proposed filter.

3.1 Experiment with Synthetic Phantom Image

Figure 1(a) shows the synthetic phantom image containing four cysts. The noisy US image, simulated using Field-II simulator [13, 14], is shown in Fig. 1(b). Parameters of the considered filters are optimized to result in the least value of MSE. The optimized parameter values along with the MSE, FoM and SSIM values are listed in Table 1. Since the noisy image is in log compressed domain, the synthetic phantom image is also converted into log compressed domain and used as the reference. The filtered outputs are shown in Fig. 2.

Table 1. Optimized values of the parameters for phantom image and the observed values of evaluation metrics

Over the uniform intensity regions the gradients are influenced by stochastic behaviour of noise. The inconsistency in the gradients of neighbouring pixels helps the proposed filter to increase homogeneity in every direction, resulting in the reduction of noise. On the other hand, the pixels near the object boundaries or the edge pixels have consistent gradient orientation, therefore, the homogeneity is increased only in the direction of the edge. As a result, the proposed filter provides the best values of FoM and SSIM. Further, the least value of MSE reflects its filtering ability. The proposed filter shows 42% improvement in FoM and 10.46% improvement in MSE, as compared to the best result achieved by existing filters. Among the existing filters, DPAD and SBF result in decent values of MSE and SSIM, however, the quality of their filtered outputs (Fig. 2) is relatively poor. The object boundaries are highly distorted and irregular. An important thing to note here is that the proposed filter shows considerable improvement over the conventional GF.

Fig. 3.

Cardiac US image with marked regions

Fig. 4.

Filtered images obtained using (a)–(d) DPAD, OBNLM, SBF, and the proposed filter.

3.2 Experimentation with Real Ultrasound Image

A real cardiac US image is acquired using GE Healthcare Vivid US system with full consent of the subject. The evaluation metrics used in synthetic image experiment require noiseless reference image. Therefore, a different metric known as contrast to noise ratio (CNR) is used here. It is defined as:

$$\begin{aligned} \text {CNR}=\frac{\vert \mu _{C_1}-\mu _{C_2}\vert }{\sqrt{\sigma ^2_{C_1}+\sigma ^2_{C_2}}} \end{aligned}$$

(21)

where $\mu _{C_r}$ and $\sigma ^2_{C_r}$ are the mean and variance of the class $C_r$ representing object or background. High value of CNR represents the high contrast. CNR measurement requires two regions of similar sizes representing different classes, tissue and background. Two such regions are identified in the image considered for experiment, shown in Fig. 3. Region 1 represents blood chamber and region 2 contains septum wall. The filtered images are shown in Fig. 4. The optimized parameter values are listed in Table 2 along with the measured CNR. Lee filter and GF are not considered due to their poor performances in synthetic phantom image experiment. SBF results in the best value of CNR, however the quality of filtered output is poor. Surprisingly the OBNLM provides better image quality despite having the lowest CNR value. On the other hand, the proposed filter gives balanced output. It results in the second best value of CNR and also provides good quality image.

Table 2. Optimized values of the implementation parameters for the real cardiac US image shown in Fig. 3.

Fig. 5.

(a), (c) and (e) Real liver US images, (b), (d) and (f) corresponding filtered outputs obtained using the proposed filter.

The proposed filter is further tested on a real US liver images obtained from a publically available database¹. The liver images and the corresponding filtered outputs obtained using the proposed filter are shown in Fig. 5. It is clear that the noise is efficiently removed while preserving the important structural details.

4 Conclusions

In this paper an edge aware GF is proposed. The filter uses edge characteristics inside the GF framework. It allows the filter to iteratively remove speckle while preserving the structural details. The filter requires almost no parameter tuning which is a big challenge faced by the existing filters. The filter provides best results for MSE, FoM and SSIM and shows competitive performance in terms of CNR. The obtained filtered images show that the proposed filter is appropriate for US image enhancement.