1 Introduction

Digital image has been widely used in our daily life. However, during the acquiring process, it is inevitably corrupted by the degraded factors, mainly including the precision in measurements of sensors, motion blur, lens aberration. Therefore, in order to obtain the high quality images, there has been a growing attention in image denoising techniques.

Generally speaking, given a corrupted image u 0 = A u + ε, where A stands for a linear blurring operator, ε is the additive white Gaussian noise and u is the original image, restoring u from u 0 often is an ill-posed inverse problem. In the past decades, a great variety of image recovery methods for solving the inverse problems have been presented, such as the regularization methods based [8, 12, 27, 31], the image sparse representation based [3, 6, 15, 28], the mixture models learning based [19, 25, 32], and so on. The total variation (TV) regularization [18, 29] as a classical image restoration method, utilizes priors on certain distributions of image gradients to locally regularize image, which enables to obtain satisfying results on piecewise-smooth images like cartoon images. The sparse representation method for image restoration is based on the fact that image can be sparsely represented on a redundant dictionary adaptively learned, and has proven its validity in recovering a wide variety of images [17, 30], including the natural images, the medical images, the aerial images and the satellite images. However, the traditional sparse representation method assumes that the atoms in dictionary are independent of each other and the structure information between the atoms is not taken into consideration. So as to achieve more satisfying denoised results, at present, many people have devoted to the research on the correlation among the dictionary atoms and propose a number of structured sparse representation methods.

The mixture model learning based image denoising method employs a small number of mixture components to learn priors over image patches for image statistical modeling, which has the advantages of the well-understood mathematical behaviors and relatively low computational complexity. Especially, the Gaussian Mixture Model (GMM) [11, 14, 16, 26] based image restoration method has proven its effectiveness and achieved good results, which actually is a kind of structured sparse representation method. Compared with the dictionary learning, learning image priors using GMM is less time consumption and easy to implement. Presently, the Expected Patch Log Likelihood (EPLL) which restores an image by maximizing the expected patch log likelihood, as one of the most important image denoising methods based on Gaussian mixture image prior has achieved a satisfying result.

In addition, the gradient information of image has been successfully introduced into the TV model, the pixel-level MAP-estimation [2] and non-local sparse representation [4, 5, 23], for preserving the small-scale textures of image. Besides, image gradient is frequently employed in image quality assessment, on account of the fact that it has the advantage of acquiring more fine structures of images. Motivated by this, we incorporate the gradient fidelity term [13] with the GMM based image denoising method for further enhancing the image denoising performs during the noise removal.

Moreover, recently, regularization parameter selection has received much attention, which has a great effect on preserving more details of image when denoising. At present, numerous methods for regularization parameter selection have been put forward, including the discrepancy principle based [21], the residual image statistics (RIS) based [7], the L-curve and gradient based [24]. In this paper, we focus on the image gradient based regularization parameter selection problem for GMM based image denoising method. Unfortunately, the image gradient is sensitive to noise and can not acquire satisfying results when the image is corrupted seriously. By observing that the local entropy of image has a good robustness to noise, we attempt to construct a new adaptive regularization parameter using image gradient matching and the local entropy of image. Furthermore, we analyze the relationship between the regularization parameters of the data-fidelity term and gradient-fidelity term, and propose a novel regularization parameter scheme for GMM based image denoising method for preserving more small-scale textures and details of images.

This paper is organized as follows: Section 2 briefly reviews the GMM based image denoising method, Section 3 introduces our proposed method with adaptive regularization parameters. In Section 4 we show experimental results and Section 5 give a simple conclusion.

2 GMM based image restoration model

Image u including N pixels can be divided into N overlapped image patches. Let u i (i=1,2,...,N) denote an image patch with the size of\(\sqrt L\times \sqrt L\) obtained by u i = R i u, where L is the number of pixels in each image patch, R i denotes an operator for extracting image patch u i from image u at position i. Given that there exist K mixture components and the patch u i is independent of each other, then the density function of the GMM on an image patch u i can be given by:

$$ p(u_{i})=\sum\limits_{j=1}^{K} \pi_{j}N(u_{i}|\mu_{j},{\Sigma}_{j}) $$
(1)

where π j is the mixing weights, μ j and Σ j are the corresponding mean and covariance matrix, N(u i |μ j ,Σ j )is the Gaussian distribution, which is defined as:

$$ \begin{array}{ll} N(u_{i}|\mu_{j},{\Sigma}_{j})&=\frac{1}{(2\pi)^{\frac{L}{2}}}\frac{1}{|{\Sigma}_{j}|^{\frac{1}{2}}}\\ &\times exp(-\frac{1}{2}(u_{i}-\mu_{j})^{T}{\Sigma}_{j}^{-1}(u_{i}-\mu_{j}))\\ \end{array} $$
(2)

Owing to the independence of each patch, the joint conditional density of the image u=(u 1,u 2,...,u N ) can be modeled as:

$$ p(u)={\prod}_{i=1}^{N} \sum\limits_{j=1}^{K}\pi_{j} N(u_{i}|\mu_{j},{\Sigma}_{j}) $$
(3)

the log of likelihood function is written as:

$$ \begin{array}{ll} L({\Theta})&=\log p(u)\\ &=\log {\prod}_{i=1}^{N} {\sum}_{j=1}^{K}\pi_{j} N(u_{i}|\mu_{j},{\Sigma}_{j})\\ &= {\prod}_{i=1}^{N} \log {\sum}_{j=1}^{K}\pi_{j} N(u_{i}|\mu_{j},{\Sigma}_{j}) \end{array} $$
(4)

where Θ={π j ,μ j ,Σ j } is the set of the model parameters.Then the optimization can be solved easily by maximizing the (4) with the help of the Expectation Maximization algorithm (EM).

Lately, image priors as driving force for image denoising have been popular, which plays an important role in image denoising. One such a popular prior is learned by GMM. From (1), learning the priors on a given patch set with the EPLL is defined as follws:

$$ EPLL(u)=\log p(u)=\sum\limits_{i}\log p(R_{i} u) $$
(5)

Given a corrupted image u 0, restoring u from u 0 with the EPLL is equivalent to maximize the posterior probability:

$$ \begin{array}{ll} \max\limits_{u} p(u|u_{0})&=\max\limits_{u}\left\{ \log p(u_{0}|u)+\log p(u)\right\}\\ &=\max \limits_{u}\left\{ \log p(u_{0}|u)+EPLL(u)\right\}\\ &=\min \limits_{u}\left\{ -\log p(u_{0}|u)-EPLL (u)\right\} \end{array} $$
(6)

Then the image denoising model based on EPLL is derived as:

$$ \begin{array}{ll} &\min\limits_{u}\left\{ \frac{\lambda}{2}\left\|u-u_{0}\right\|^{2}-EPLL(u)\right\}\\ =&\min\limits_{u}\left\{ \frac{\lambda}{2}\left\|u-u_{0}\right\|^{2}-{\sum}_{i} \log p(R_{i}u)\right\} \end{array} $$
(7)

where λ is the regularization parameter and the (7) can be solved by the Half Quadratic Splitting algorithm [1, 9, 10, 20].

3 Proposed method with adaptive regularization parameter

Regularization parameter selection plays an important role in image restoration. As explained in [22], an adaptive regularization parameter for the fidelity term is effective to preserve fine structures of images. This is naturally owing to the parameter in fidelity terms is adaptive to the image information and will change its value automatically in different regions of the image. Following the above motivation, we construct a new adaptive parameter coupling local entropy of images and analyze the relationship between the regularization parameters of the data-fidelity term and gradient-fidelity term. In this paper, for the sake of acquiring more satisfying denoised results, a novel Gaussian mixture model based image denoising method with adaptive fidelity term is presented as follows:

$$ \min\limits_{u}\{ \frac{\lambda(x,y)}{2}\left\|u-u_{0}\right\|^{2}+\frac{\alpha(x,y)}{2}\left\|\nabla u-\nabla (G_{\sigma} * u_{0})\right\|^{2}\\ -\sum\limits_{i}\log p(R_{i}u)\} $$
(8)

where ∇ denotes the image gradient, G σ is a Gaussian filter operator, ∗ is the convolution symbol, the first term imposes a global constraint on the restored image; the second term is the gradient fidelity term which describes the similarities in gradient between the corrupted images and the restored ones for preserving more image details. λ and α are the weight coefficients and can be chosen as a function of the gradient and local entropy of corrupted image in the following form:

$$ \alpha(x,y)=\frac{1}{1+(|\nabla u|\cdot f(E(x,y))/k_{0})^{g(|\nabla u|)}} $$
(9)
$$ \lambda(x,y)=k(1-\alpha(x,y)) $$
(10)

where k is a given constant, k 0 is a threshold value, g(|∇u|) is designed by:

$$ g(|\nabla u|)=2+\frac{k_{1}}{(1+|\nabla u|/k_{0})^{2}} $$
(11)

where k 1 is also a threshold value, when |∇u| is large, \(g(|\nabla u|)\rightarrow 2\), when \(|\nabla u|\rightarrow 0 \), \(g(|\nabla u|)\rightarrow 2+k_{1}\), E(x,y) is the local entropy of image and is defined by:

$$ E(x,y)=-\sum\limits_{i=1}^{S-1}p_{i}\log p_{i} $$
(12)

where p i = n i /(M×N) is the probability of the i-th gray level, S is the maximum gray level in the neighborhood centered on the pixel (x,y), in practice, we generally set the size of the local neighborhood as 3×3 and n i is the number of pixels with the i-th gray level, M×N is the size of image and f(E(x,y)) is defined as follows:

$$ f(E(x,y))=1+\frac{E(x,y)-\min(E(x,y))}{\max(E(x,y))-\min(E(x,y))}G $$
(13)

where G is the maximum of image gradient norm, and the range region of f(E(x,y)) is [1,G+1], which alleviates the impact of image gradient on the regularization parameters due to the introduction of the local entropy of images and makes the adaptive parameters have a better robustness.

The regularization parameter λ(x,y) and α(x,y) vary with different regions of the image. λ(x,y) is set to be small in the smooth regions of image, while α(x,y) is large so as to remove much noise in the meanwhile guarantee the similarity between the restored image and the corrupted one. At the edge of image, α(x,y) is small but λ(x,y) is large, for the sake of preserving more edges of the image. Meanwhile, the given k can help balance the regularization parameters λ(x,y) and α(x,y), and make them have proper values in different regions of the image for preserving more details of image while smoothing noises. Consequently, an adaptive fidelity term is useful and effective for preserving more textures and details of image in the process of noise removing.

Here, we employ the Half Quadratic Splitting algorithm to solve (8). The (8) is equivalently transformed into the following function by introducing a set of auxiliary variables {z i } as follows:

$$ \begin{array}{ll} \min\limits_{u,\{z_{i}\}}&\{\frac{\lambda(x,y)}{2}\left\|u-u_{0}\right\|^{2}+\frac{\alpha(x,y)}{2}\left\|\nabla u-\nabla (G_{\sigma} * u_{0})\right\|^{2}\\ &+{\sum}_{i}\{\frac{\beta}{2}(\left\|R_{i}u-z_{i}\right\|^{2})-\log p(z_{i})\}\} \end{array} $$
(14)

where β is the penalty parameter which usually is set to be large enough to ensure that the solution of (14) is close to that of (8).

For solving (14), at first, we choose the most likely Gaussian mixing weight j m a x for each patch R i u:

$$ \begin{array}{ll} j_{max}&=\max\limits_{j} p(j|R_{i}u)\\ &=\max\limits_{j}p(R_{i}u|j)p(R_{i}u)\\ &=\max\limits_{j} \{\log p(R_{i}u|j)+\log p(R_{i}u)\} \end{array} $$
(15)

Accordingly, the image patch has the mean \(\mu _{j_{max}}\) and covariance matrix \({\Sigma }_{j_{max}}\), then (14) is minimized by alternatively updating z i and u.

For a fixed u n, updating z i is equivalent to solve the local MAP-estimation problem as follows:

$$ \min\limits_{z_{i}}\{ \frac{\beta}{2}\left\|R_{i} u-z_{i}\right\|^{2}-\log p(z_{i})\} $$
(16)

It is in fact a Wiener solution:

$$ z_{i}^{n+1}=({\Sigma}_{j_{max}}+\frac{1}{\beta}I)^{-1}\cdot(R_{i}{u^{n}}{\Sigma}_{j_{max}}+\frac{1}{\beta}\mu_{j_{max}}I) $$
(17)

where I is the identity matrix.

For a fixed z i , using the Euler-Lagrange formula for (14) and the corresponding Euler equation of it is:

$$ \lambda(x,y)(u-u_{0})+\sum\limits_{i}\beta {R_{i}^{T}}(R_{i}u-z_{i})-\alpha(x,y){\Delta} u+\alpha(x,y)(G_{\sigma}*\nabla u_{0})=0 $$
(18)

the corresponding PDE as:

$$ \frac{\partial u}{\partial t}=\lambda(x,y)(u-u_{0})-\sum\limits_{i}\beta {R_{i}^{T}}(R_{i}u-z_{i})+\alpha(x,y){\Delta} u-\alpha(x,y)(G_{\sigma}*\nabla u_{0}) $$
(19)

It can be solved by the gradient descent algorithm and updating u as follows:

$$ \begin{array}{ll} u^{n+1}&=u^{n}+{\Delta} t [\lambda(x,y)(u_{0}-u^{n})-{\sum}_{i}\beta {R_{i}^{T}}(R_{i}u^{n}-{z_{i}^{n}})+\alpha(x,y)(u^{n}_{xx}+u^{n}_{yy})\\ &-\alpha(x,y)(G_{\sigma}*(u_{0xx}+u_{0yy})] \end{array} $$
(20)

where Δt is the time step, u x x ,u y y ,u 0x x ,u 0y y are respectively the second-order partial derivative of u and u 0.

In summary, our suggested algorithm can be implemented as follows:

  1. Step1.

    Input corrupted image u 0 , model parameters β,Δt and iteration stop- ping tolerance ε, initialize regularization parameters λ,α ;

  2. Step2.

    Choose the most likely Gaussian mixing weights j m a x for each image patch z i with the model (15);

  3. Step3.

    Calculate \({z_{i}^{1}}\) using (17);

  4. Step4.

    Calculate u 1 using (20) with updated regularization parameters λ and α;

  5. Step5.

    Calculate \(z_{i}^{n+1}\) using (17);

  6. Step6.

    Pre-estimate image u n+1 using (20) with updated λ and α;

  7. Step7.

    Repeat Steps 5-6 until satisfying stopping criterion.

4 Implementation and experiment results

In our experiments, the GMM with 200 mixture components is learned from a set of 2×106 images patches sampled from the Berkeley Segmentation Database Benchmark (BSDS300) with their DC removed. Comparing our proposed method with the original EPLL and the EPLL with Xie adaptive λ and α in [22], the parameters in our proposed method are as follows:the image patch size \(\sqrt L=8\), the weighted coefficients \(\beta =\frac {1}{\sigma ^{2}}*[1 \quad 4 \quad 8 \quad 16]\), the noise standard variance σ=25 and the size of local entropy of the image is set as 3×3, the constant \( k_{0}=\frac {1}{\sigma ^{2}}\), k 1=7 and \( k=\frac {L}{2\sigma ^{2}}\).

Figure 1 demonstrates the denoised results of the original EPLL and our proposed method with adaptive λ,α on Test1 image (i.e,No.3096). The related quantitative comparison, in terms of peak signal to noise ratio (PSNR) and signal to noise ratio (SNR), are shown in Table 1. The Fig. 1a and b are respectively the original image and noisy image. The Fig. 1c shows that the denoised result obtained by the original EPLL, and we can see that some regions of the image are not smooth, in contrast, the Fig. 1d obtained by our method shows a better result. This is probably due to the fact that our proposed method incorporates the gradient fidelity term with the EPLL and the regularization parameters are adaptive to the image information, which can help preserve more details of image and make the degraded image smoother during the denoising procedure. In addition, as demonstrated in Table 1, the PSNR value and SNR value of our method is higher than the original EPLL.

Fig. 1
figure 1

Denoising results on Test1 image, a-b Original and noisy images, respectively. c-d Denoising results of EPLL and proposed method, respectively

Full size image
Table 1 The PSNR and SNR results of different denoising models
Full size table

In Fig. 2. we also compare our proposed method with the original EPLL on Test2 image (i.e,No.3063). The related quantitative comparison, in terms of PSNR and SNR are shown in Table 2. From the denoised results shown in Fig. 2c and d, we can see that our proposed method outperforms the original EPLL and make the image smoother. In addition, we enlarge the middle part of the denoised result and put it on the right of image. From the Fig. 2d, we can find that our proposed method can better preserve the edges and small-scale textures of the image. This is because our proposed method has adaptive regularization parameters and can help preserve more details of images. Therefore, by comparison, our proposed method can obtain visually satisfying results and performs better in PSNR and SNR.

Fig. 2
figure 2

Denoising results on Test2 image, a-b Original and noisy images, respectively. c-d Denoising results of EPLL and proposed method, respectively

Full size image
Table 2 The PSNR and SNR results of different denoising models
Full size table
Fig. 3
figure 3

Denoising results on Barbara image, a-b Original and noisy images, respectively. c-d Denoising results of EPLL with fixed λ,α and proposed method, respectively

Full size image

Figure 3. demonstrates the denoised results of our proposed method and the comparison with the EPLL with fixed λ,α on Barbara image. The corresponding PSNR and SNR are shown in Table 3. We enlarge the right shoulder of denoised result and put it on the right of image. From the result in Fig. 3c we can see that some small-scale textures of the image are not clear, while the result of our proposed method in Fig. 3d preserve more textures. This is probably on account of the fact that the regularization parameters being a constant of the method in Fig. 3c, while the parameters of our proposed method vary with different regions of the image. They can change their values automatically according to the image information and help preserve more fine structures in image. Therefore, by comparing, our proposed adaptive method outperforms the EPLL with a fixed λ,α both in PSNR and SNR.

Table 3 The PSNR and SNR results of different denoising models
Full size table
Fig. 4
figure 4

Denoising results on Test3 image, a-b Original and noisy images, respectively. c-d Denoising results of EPLL with Xie λ,α and proposed method, respectively

Full size image
Table 4 The PSNR and SNR results of different denoising models
Full size table

Figure 4.demonstrates the denoised results of the EPLL with Xie λ,α and our proposed with adaptive λ,α on the Test3 image (i.e,No.160068). The related PSNR and SNR are shown in Table 4. The Fig. 4a and b are respectively the original image and noisy image. The Fig. 4c shows that the denoised result obtained by the EPLL with Xie adaptive λ,α and the Fig. 4d shows that the denoised result of our proposed method. We construct a new adaptive regularization parameter with local entropy of image. Due to the robustness of local entropy to noise, our proposed method can not only remove noise but preserve the weak edges and fine details of image. Thus, we can see that our proposed method can obtain a more satisfying result.

5 Conclusions

In this paper, a novel GMM based image denoising method with gradient fidelity term has been proposed, which can help preserve more small-scale textures and details of images during the noise removal. The GMM is a powerful tool for learning image priors, that is easy to implement and requires a small amount of parameters to estimate. In addition, compared with the dictionary learning in sparse representation, GMM has the advantages of relatively low computational complexity and the well-understood mathematical behavior. Furthermore, for preserving more weak edges of images when denoising, we construct a new adaptive selection scheme for the regularization parameters by means of the local entropy of the image, which varies with different regions of the image and has a good robustness to noise. Experiments show that our proposed method shows a clear improvement in comparison to the original EPLL algorithm and EPLL with fixed regularization parameters method for image denoising both visually and quantitatively.