Keywords

1 Introduction

Image restoration, especially image denoising, is a very important process and is often necessary as a pre-processing for other imaging techniques such as segmentation and compression. Images occur in many forms in our life. Among them, ultrasound provides low cost images helpful for medical diagnosis. But since these images are corrupted by speckle noise, it is not possible to do automatic processing.

In general, an observed image f, corrupted by Gaussian noise n, is represented by the equation

$$ f = u + n, $$
(1)

where u is the original noise free image. Here \( u,f :\;\varOmega \subset {\mathbb{R}}^{2} \to {\mathbb{R}} \). For any denoising model, the main objective is to reconstruct u from an observed image f. For the last two decades, various partial differential equation (PDE) based models have been developed for this purpose [1, 4–6, 10–13, 15, 19–21, 23].

In 1992, Rudin et al. [21] proposed the total variation (TV) denoising model. But TV model is known for its staircasing effects. In 2005, Buades, Coll and Morel proposed the filtering based nonlocal means model [3] which did not provide the staircasing effect and was more accurate than the TV model. But this method was very slow. Later in 2008, Gilboa and Osher introduced nonlocal PDE operators [7–9, 14] and Bresson used it to develop a nonlocal version of the TV model [2, 22].

But these models were all meant for Gaussian or additive noise images. Speckle noise is multiplicative and needs to be treated differently. The first effective speckle noise model was introduced by Krissian et al. in 2005 [16, 17]. In 2007, Lim and Williams [18] proposed a realistic version of the Krissian model. In this paper, we develop a nonlocal speckle denoising model based on the model introduced by Lim and Williams [18].

2 PDE Based Models

In 1992, Rudin et al. [21] proposed the total variation (TV) denoising functional, for Gaussian noise images (1), as:

$$ F(u) = \int\limits_{\varOmega } {|\nabla u|\;{\text{d}}x + \frac{\lambda }{2}} \int\limits_{\varOmega } {\left( {f - u} \right)^{2} {\text{d}}x.} $$
(2)

where \( \sigma \) is the standard deviation of the noise n and \( \lambda \) is a constraint parameter. The equivalent Euler-Lagrange equation gives the TV denoising model as

$$ \frac{\partial u}{\partial t} - \nabla \cdot \left( {\frac{\nabla u}{|\nabla u|}} \right) = \lambda (f - u). $$
(3)

To avoid singularities, it was regularized by using \( |\nabla u| \approx |\nabla^{\varepsilon } u| = (u_{x}^{2} + u_{y}^{2} + \varepsilon^{2} )^{1/2} \).

The TV model was applicable only on Gaussian noise or additive noise. But ultrasound or SAR images contain speckle noise which is multiplicative and cannot be removed using the Gaussian noise models. In 2005, Krissian et al. [16] proposed the most effective speckle noise model based on the TV model. The authors considered the speckle noise equation by

$$ f = u + \sqrt u n, $$
(4)

where u is the desired image to find, n is Gaussian noise, and f denotes the observed image. Thus, we have \( n = \frac{f - u}{\sqrt u } \). By replacing f − u in (2) with the new expression for n, the minimization functional for speckle denoising is given by

$$ F(u) = \int\limits_{\varOmega } {\left[ {|\nabla u| + \frac{\lambda }{2}\left( {\frac{f - u}{\sqrt u }} \right)^{2} } \right]\;{\text{d}}x.} $$
(5)

From energy minimization of this functional, the TV-based speckle denoising model can be derived as:

$$ \frac{\partial u}{\partial t} - \frac{{u^{2} }}{f + u}|\nabla^{ \epsilon } u| \nabla \cdot \left( {\frac{\nabla u}{{|\nabla^{\varepsilon } u|}}} \right) = \lambda |\nabla^{ \epsilon } u| (f - u). $$
(6)

It was later noted by Lim and Williams [18], that the coefficient of the diffusion term in (6) is given by

$$ |\nabla u| \nabla \left( {\frac{\nabla u}{|\nabla u|}} \right) \approx \frac{{u^{2} }}{f + u} \approx u/2, $$

assuming that \( f \approx u \). Hence, this coefficient makes the diffusion faster in the lighter region (where the image values are high) and slower in the darker region (where the image values are low). Since diffusion is associated with noise removal, noise is removed or reduced from the diffusion process due to the second term in Eq. (6). In 2007, Lim and Williams [18] proposed the following realistic noise model

$$ f = u + (\sqrt {u - f_{s} } )n, $$
(7)

where f s denotes smoothed version of the noisy image f. Using this noise equation, the non-standard anisotropic diffusion (NSAD) model is derived as:

$$ \frac{\partial u}{\partial t} - C|u - f_{s} |^{\alpha } |\nabla^{ \epsilon } u|^{2 - p} \nabla \cdot \left( {\frac{\nabla u}{{|\nabla^{\varepsilon } u|^{2 - p} }}} \right) = \lambda |\nabla^{ \epsilon } u|^{2 - p} (f - u), $$
(8)

where, C is a constant calculated numerically. Here, \( 1/2 < \alpha < 2 \), and p is chosen between 0 and 1 so that the model (8) becomes nonconvex and it has edge enhancing features [13].

3 Nonlocal PDE Based Models

Apart from the PDE based models, there were also filtering based denoising models. The neighborhood filter has a big advantage of not blurring the edges. Based on this fact, Buades, Coll, and Morel proposed the new state of art image denoising algorithm known as nonlocal means algorithm [3]. The algorithm is given by the formula

$$ NL[u](x) = \frac{1}{C(x)}\int\limits_{\varOmega } {e^{{ - \frac{{(G_{a} { \star }|u(x + \cdot ) - u(y + \cdot )|^{2} )(0)}}{{h^{2} }}}} u(y)\;{\text{d}}y,} $$
(9)

where \( C(x) = \int_{\varOmega } e^{{ - \frac{{(G_{a} { \star }|u(x + \cdot ) - u(z + \cdot )|^{2} )(0)}}{{h^{2} }}}} \;{\text{d}}z \).

Here, G a is the Gaussian kernel with standard deviation a, h is a filtering parameter and \( u(x + \cdot ) \) denotes the neighborhood of the pixel \( x \). However in this case, each pixel value is denoised using the weighted average of all the pixels in the image. Thus for a given discrete noisy image \( u = \{ u(i):i \in I\} \), the estimated value \( NL[u](i) \), for a pixel \( i \), is computed as

$$ NL[u](i) = \sum\limits_{j \in I} w(i,j)u(j), $$
(10)

where the weight w(i, j) depends on the similarity of the i and j pixels and satisfies the conditions \( 0 \le w(i,j) \le 1 \) and \( \sum\nolimits_{j} w(i,j) = 1 \). The weight function is defined as

$$ w(i,j) = \frac{1}{C(i)}e^{{ - \frac{{||u({\mathcal N}_{i} ) - u({\mathcal N}_{j} )||_{2,a}^{2} }}{{h^{2} }}}} , $$
(11)

where C(i) is the normalizing constant given by

$$ C(i) = \sum\limits_{j} e^{{ - \frac{{||u({\mathcal N}_{i} ) - u({\mathcal N}_{j} )||_{2,a}^{2} }}{{h^{2} }}}} . $$
(12)

The similar neighborhoods have a very small Euclidean distance which in turn results in larger weight, since the weight function is decreasing with respect to Euclidean distance. Here the parameter h acts as a degree of filtering. For computational purposes, the authors restricted the search of similar windows to a window of size 21 × 21 pixels and the similarity square neighborhood \({\mathcal{N}}\) i of 7 × 7 pixels. Though the method was very good in removing noise, it is very slow and hence quite impractical.

In 2005, Kindermann et al. [14] first tried to relate the filtering method to the energy minimization. In 2008, Gilboa and Osher [9] extended some known PDE and variational techniques in image processing to the nonlocal framework. The biggest challenge for the extension was the fact that the classical derivatives are local operators. Thus, they had to redefine the required operators following the ideas of Zhou and Schlkopf [24, 25]. In this paper, we would like to introduce a nonlocal speckle denoising model based on the NSAD model [18]. The NSAD model is known to provide sharper edges than TV. Hence, we apply the nonlocal PDE operators it is expected to provide even better results for the existing nonconvex model.

4 NLAD Model

According to our observation, the speckle model by Krissian et al. is quite effective and NSAD model is a realistic one. Keeping this in mind, we develop a new model incorporating the positive aspects of both. Here, we propose the following form of a noise model:

$$ f = u + \left( {\sqrt {u + C|u - f_{s} |^{\alpha } } } \right)n, $$
(13)

where f is the observed image or initial image, u is the desired image to find, n is Gaussian noise, and f s denotes smoothed version of the noisy image f. This gives

$$ n = \frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}. $$
(14)

Now, the regularized functional will be given by

$$ \mathop {\hbox{min} }\limits_{u} F_{ \in } (u) \equiv \mathop {\hbox{min} }\limits_{u} \left( {\int\limits_{\varOmega } {\left[ {|\nabla^{ \in } u| + \frac{\lambda }{2}\left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} } \right]\;{\text{d}}x} } \right). $$
(15)

We apply Euler-Lagrange to minimize this functional. Following the NSAD model [18] closely, we derive an edge enhancing accelerated diffusion (EEAD) model based on the nonconvex model as:

$$ \frac{\partial u}{\partial t} = \left( {u + C|u - u_{s} |^{\alpha } } \right)|\nabla^{ \epsilon } u| \nabla \cdot \left( {\frac{\nabla u}{{|\nabla^{ \epsilon } u|}}} \right) + \lambda |\nabla^{ \epsilon } u| (f - u), $$
(16)

where \( \alpha < 1 \) and C is a constant calculated numerically. Now we extend this model to a nonlocal framework based on the nonlocal PDE operators introduced by Gilboa and Osher [9] and obtain the following nonlocal accelerated diffusion (NLAD) model:

$$ F(u) = \int\limits_{\varOmega } {\left[ {|\nabla_{\text{NL}} u| + \lambda \left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} } \right]\;{\text{d}}x.} $$
(17)

In the next section, we describe a numerical procedure for this model. It is based on the Split Bregman scheme.

5 Split Bregman Scheme for NLAD Model

We now describe the Split Bregman scheme to solve the NLAD functional (17). The Split Bregman algorithm was introduced by Goldstein and Osher in [xxx1]. This was developed as a faster numerical scheme for the nonlocal TV denoising problem. The authors applied operator splitting and then used the Bregaman iteration to solve the resulting constrained minimization problem. It was used to find solution to a nonlocal version of the regular TV model [2, 22]. The same scheme was also used to find a nonlocal version of the TV based speckle denoising model [xxx2] introduced by Krissian [16, 17]. We will closely follow the steps described in these papers and develop a numerical scheme for the NLAD model. First we develop the Split Bregman scheme for the EEAD model and then extend the idea to a nonlocal framework.

5.1 Split Bregman Scheme for EEAD Model

As discussed in Sect. 4, the minimization functional for the EEAD model was given by (15). For Split Bregman scheme, we introduce \( d = \nabla u \) and construct the constrained minimization problem as

$$ \mathop {\hbox{min} }\limits_{d,u} \int\limits_{\varOmega } {|d| + \frac{\lambda }{2}\left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} \;{\text{d}}x,\quad {\text{subject}}\;{\text{to}}\;d = \nabla u.} $$
(18)

This can be changed to a unconstrained problem as:

$$ \mathop {\hbox{min} }\limits_{u} F(u), $$

where

$$ F(u) = \int\limits_{\varOmega } |d| + \frac{\lambda }{2}\left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} \;{\text{d}}x + \frac{\beta }{2}{\parallel }d - \nabla u - b{\parallel }_{2}^{2} . $$
(19)

Here, \( \beta \) is a penalty parameter and b is the Bregman iteration variable. We can now split this in two subproblems of u and d given as:

$$ \mathop {\hbox{min} }\limits_{u} \int\limits_{\varOmega } \frac{\lambda }{2}\left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} \;{\text{d}}x + \frac{\beta }{2}{\parallel }d - \nabla u - b{\parallel }_{2}^{2} , $$
(20)
$$ \mathop {\hbox{min} }\limits_{d} \int\limits_{\varOmega } |d| \;{\text{d}}x + \frac{\beta }{2}{\parallel }d - \nabla u - b{\parallel }_{2}^{2} . $$
(21)

Now, using the following approximation

$$ \frac{\partial }{\partial u}\left( {\frac{\lambda }{2}\left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} } \right) = 2\lambda \frac{u - f}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}. $$
(22)

Therefore, the optimality condition for u in (20) gives us

$$ (\lambda - \frac{\beta }{2}\left( {u + C|u - f_{s} |^{\alpha } } \right)\varDelta )u = \lambda f - \frac{\beta }{2}\left( {u + C|u - f_{s} |^{\alpha } } \right){\text{div}} (d - b). $$
(23)

The optimality condition can be discretized using the definition of discrete gradient, divergence and Laplacian as discussed in [9]. This gives

$$ \begin{aligned} \left( {\lambda + 2\beta \left( {u_{ij} + C|u_{ij} - f_{s,ij} |^{\alpha } } \right)} \right)u_{ij} & = \lambda f_{ij} + \beta \left( {u_{ij} + C|u_{ij} - f_{s,ij} |^{\alpha } } \right) \\ & \quad \times \left( {u_{ij} + C|u_{ij} - f_{s,ij} |} \right)(u_{i + 1,j} + u_{i - 1,j} + u_{i,j + 1} + u_{i,j - 1} \\ & \quad + d_{x,i - 1,j} - d_{x,ij} + d_{y,i,j - 1} - d_{y,ij} - b_{x,i - 1,j} + b_{x,ij} - b_{y,i,j - 1} + b_{y,ij} ). \\ \end{aligned} $$
(24)

Thus, we get

$$ \begin{aligned} u_{ij}^{k + 1} & = \frac{1}{{\left( {\lambda + 2\beta \left( {u_{ij} + C|u_{ij} - f_{s,ij} |^{\alpha } } \right)} \right)}}[\lambda f_{ij} + \beta \left( {u_{ij} + C|u_{ij} - f_{s,ij} |^{\alpha } } \right) \\ & \quad \times \left( {u_{ij} + C|u_{ij} - f_{s,ij} |} \right)(u_{i + 1,j} + u_{i - 1,j} + u_{i,j + 1} + u_{i,j - 1} \\ & \quad + d_{x,i - 1,j} - d_{x,ij} + d_{y,i,j - 1} - d_{y,ij} - b_{x,i - 1,j} + b_{x,ij} - b_{y,i,j - 1} + b_{y,ij} ]. \\ \end{aligned} $$
(25)

Since there is no coupling between elements of d, the optimal value of d is computed using the shrinkage operator described in [xxx3] and [xxx4]:

$$ d_{ij}^{k + 1} = \frac{{\nabla u_{ij}^{k + 1} + b_{ij}^{k} }}{{|\nabla u_{ij}^{k + 1} + b_{ij}^{k} |}}\hbox{max} \{ |\nabla u_{ij}^{k + 1} + b_{ij}^{k} | - 1/\beta ,0\} . $$
(26)

The variable b is initialized to zero and is updated after each Bregman iteration as:

$$ b_{ij}^{k + 1} = b_{ij}^{k} + \nabla u_{ij}^{k + 1} - d_{ij}^{k + 1} . $$
(27)

The scheme (25)–(27) provides a much faster solution for the TV based EEAD model.

5.2 Split Bregman Scheme for NLAD Model

For our new NLAD model (17) the Split Bregman functional will be of the form

$$ F(u) = \int\limits_{\varOmega } \left[ {|\nabla_{\text{NL}} u| + \lambda \left( {\frac{f - u}{{\sqrt {u + C|u - f_{s} |^{\alpha } } }}} \right)^{2} } \right] \; {\text{d}}x. $$
(28)

Then the optimality condition for u gives us the equation

$$ \left( {\lambda - \beta (u + C|u - f_{s} |^{\alpha } )\varDelta_{\text{NL}} } \right)u = \lambda f - \beta (u + C|u - f_{s} |^{\alpha } ){\text{div}}_{\text{NL}} (d - b). $$

Denoting the discretized points \( x,y \in \varOmega \times \varOmega \) by i and j, and using the discrete definition for \( {\text{div}}_{\text{NL}} \) and \( \varDelta_{\text{NL}} \) [2, 9], the discrete minimization scheme is given as:

$$ \begin{aligned} & \lambda u_{i} - \beta (u_{ij} + C|u_{ij} - f_{s,ij} |^{\alpha } )\left( {\sum\limits_{j} w_{ij} (u_{j} - u_{i} )} \right) \\ & \quad = \lambda f_{i} - \beta (u_{ij} + C|u_{ij} - f_{s,ij} |^{\alpha } )\left( {\sum\limits_{j} \sqrt {w_{ij} } (d_{ij} - d_{ji} - b_{ij} + b_{ji} )} \right). \\ \end{aligned} $$
(29)

The iteration steps for this method is given by

$$ \begin{aligned} u_{i}^{k + 1} & = \frac{1}{{\lambda + \beta (u + C|u - f_{s} |^{\alpha } ) \sum_{j} w_{ij} }}[\beta (u + C|u - f_{s} |^{\alpha } ) \sum\limits_{j} w_{ij} u_{j}^{k} + \lambda f_{i} \\ & \quad - \beta (u + C|u - f_{s} |^{\alpha } )(\sum\limits_{j} \sqrt {w_{ij} } (d_{ij} - d_{ji} - b_{ij} + b_{ji} ))], \\ \end{aligned} $$
(30)
$$ \begin{aligned} d_{ij}^{k + 1} & = \frac{{\sqrt {w_{ij} } (u_{j}^{k + 1} - u_{i}^{k + 1} ) + b_{ij}^{k} }}{{\sqrt {\sum_{j} w_{ij} (u_{j}^{k + 1} - u_{i}^{k + 1} )^{2} +{ {b_{ij}^{k}}}^{2}} }} \\ & \quad \times \hbox{max} \left( {\sqrt {\sum\limits_{j} w_{ij} (u_{j}^{k + 1} - u_{i}^{k + 1} )^{2} + b_{ij}^{k2} } - \frac{1}{\beta },0} \right), \\ \end{aligned} $$
(31)
$$ b_{ij}^{k + 1} = b_{ij}^{k} + \sqrt {w_{ij} } (u_{j}^{k + 1} - u_{i}^{k + 1} ) - d_{ij}^{k + 1} . $$
(32)

The three Eqs. (30)–(32) provide a numerical scheme for the NLAD model.

6 Numerical Results

The numerical results of the images are displayed here. We have considered one synthetic speckle noise image (lenna) and one ultrasound image (gallstone). Both images are of size 128 × 128 and the summary of results are given in Table 1. We have provided both inverted absolute residuals \( (|f - u|) \) and speckle noise residuals \( (n = \frac{f - u}{\sqrt u }) \) for both images. It is evident that the NLAD model works best among the models compared here. The EEAD model is also comparable to this one. The Krissian model is not denoising well and also shows some artificial textures in some of the resulting images. We have results for exactly same number of iterations for EEAD and Krissian models and EEAD has better results. For the NLAD model, only 1–2 iterations were enough for both images.

Table 1 NLAD model comparison for speckle noise
Full size table

From the lenna image in Fig. 1a, we see that the Krissian residual shows the presence of edges and fine textures due to the blurring effect of TV. Nonlocal means works better on the edges but not on the textures. Both EEAD and NLAD maintain texture better though EEAD has some blurring effect. In Fig. 1b, the noise residuals of lenna image show that Krissian and nonlocal means do not pick up noise very well. Both EEAD and NLAD remove noise better than the other models compared here. NLAD works best among all the models compared here. It is also evident from PSNR values listed in Table 1. The NLAD model has the highest PSNR value with EEAD being the closest to it. Krissian has the lowest PSNR. But this model is the fastest of all. Nonlocal means is the worst model among all here. It does not remove noise very well and also takes a long time. NLAD takes less than half time than nonlocal means.

Fig. 1
figure 1

NLAD model results for synthetic lenna image. a Absolute residual. b Noise residual

Full size image

The ultrasound gallstone image in Fig. 2a also shows that NLAD has the best performance in removing speckle noise. Here, also Krissian model does not preserve edges and texture, and also makes the image blurry. EEAD is removing noise quite well but the image is still blurry. Nonlocal means is not making the image blurry but it is not removing much noise. The NLAD is best in removing the noise. It is also evident from Fig. 2b. Here also we can see that NLAS picks up most noise while preserving the edges and texture. Since this is a natural image we cannot compare the PSNR values. But we do get same results for the execution time. Once again Krissian model is the fastest. The execution time for the NLAD is smaller than nonlocal means but larger than the Krissian model. Overall we can say NLAD is better and accurate than the other models compared here.

Fig. 2
figure 2

NLAD model results for ultrasound gallstone image. a Absolute residual. b Noise residual

Full size image

7 Conclusion

In this paper, we develop a nonlocal version of an edge enhancing speckle denoising model. The NLAD model provides denoising using nonlocal operators and it also presents us with accelerated diffusion. This model works really well in terms of denoising speckle noise images. We show that it picks up most noises compared to the other models discussed here. We compare this model with the TV based Krissian speckle denoising model and it is shown NLAD performs better than the Krissian model. It also works better than the EEAD model, which is basically the TV based version of the NLAD model. The biggest drawback is that it is slow compared to the TV based models. The reason behind that is the model is developed using nonlocal operators which are defined based on nonlocal means. The nonlocal means model itself is a very slow Gaussian denoising model. Apart from execution time, it is shown in this paper that nonlocal means model does not work very well with speckle noise images. The NLAD model works much slower than TV based models, but it is still faster than the nonlocal means method and gives better result than the TV based models.