1 Introduction

Digital image has extensive application foreground in our daily life. However, during image acquisition and transmission process, images are inevitably corrupted by the degraded factors, such as noise interference, motion blur and frequency aliasing. In order to obtain high quality images, image denoising is one of the most important research issues in digital image processing.

Image denoising is the problem of reducing undesired noise while preserving image details. Normally, noisy image degradation can be modeled as:u0 = u + n, where u and u0 denotes the original and noisy images, respectively, and n is the additive white Gaussian noise. Generally, estimating image u from the linear measurement is often an ill-posed inverse problem [7, 20], which can be addressed by means of nonlinear regularization technique that is closely related to the image statistic modelling. Using image prior as driving force for image restoration have always been a hot issue.

In the past several decades, many popular image priors [1, 23] have been presented, such as the gradient based [7, 20], the non-local self-similarity based [12, 25], the sparsity based [4, 10, 18, 24], and so on. Classical image restoration methods as the maximum-a-posteriori (MAP) estimation [7] and the total variation (TV) regularization [20], utilize priors on certain distributions of image gradients to locally regularize image, which are capable of obtaining good results on piecewise-smooth images like cartoon images. With the observation that similar structures are usually distributed over the whole image, the non-local self-similarity prior [1] has been usefully employed by the non-local regularization methods for texture image processing problems [12, 25]. The sparsity prior [4] is based on the fact that image patches can be sparsely represented over a redundant dictionary adaptively learned, and has proven its effectiveness in recovering a wide variety of images [10], including natural images, medical images, aerial images and satellite images.

Because of the independence assumption on the dictionary atoms, the traditional sparse representation with dictionary learning [4] has limited performance. In practice, the active atoms in the learned dictionary often exhibit strong connections. Naturally, one of promising directions for sparse representation lies in imposing more constraints on the sparse dictionary [18, 24]. Consequently, various structured sparse representation methods [11, 15, 17, 29, 36] have been developed through exploring and exploiting the structural dependences among the dictionary atoms, mainly including the group or block-sparsity based [15, 17, 29], the nonlocal self-similarity based [11, 36], and the mixture models based [13, 33]. Among these approaches, the mixture model based [13, 33] has been shown to be surprisingly competitive in handling the image restoration.

By considering that image structure within a small local window appears to be easier to model [26], the mixture model based approaches [34] employ a small number of mixture components to learn priors over image patches for image statistical modelling, which offers two advantages: relatively low computational complexity, and the well-understood mathematical behavior. Presently, compared with other mixture model [13], the Gaussian mixture model (GMM) has been popular in sparse representation based image restoration [33], for the reason that it is easy to implement and requires a small amount of parameters to estimate. Recently, the current GMM based denoising method mainly focuses on the learning strategy, such as the Expected log patch likelihood (EPLL) based method and its variants [14, 16, 21, 34]. However, how to select regularization parameter is still an open problem for the GMM based image denoising method [31].

It is well-known that regularization parameter has a great influence on the performance of denoising method. When the value of regularization parameter is too large, there will be residual noise in the restored image. When the value of regularization parameter is too small, the denoised images will probably lose important details such as the edge and textures. To date numerous methods for regularization parameter selection have been presented, including Lagrange multiplier based method [2, 6, 8, 28], the L-curve based method [19, 27], and the discrepancy principle based method [22], the structure tensor based method [32, 35] and the Scale space based method [3, 5, 30]. This work focuses on the regularization parameter selection using Lagrange multiplier for EPLL based image denoising. The above-mentioned methods can be roughly classified into two categories: global method and locally adaptive method. In general, Lagrange multiplier based methods, the discrepancy principle based method and L-curve based methods often select a single regularization parameter in a global way for image processing. Although these approaches are comparatively easy to implement, theirs performances on image denoising are unsatisfying. Structure tensor based methods attempt to employ the local image analysis tool to adaptively estimate regularization parameters. In other words, each pixel is assigned a different parameter. However parameter estimation using structure tensor fails to obtain good result for weak-edge images. The scale space based methods can be further divided into direct method and indirect method. These methods locally compute regularization parameters through scale space statistics. However regularization parameter estimation using space scale technique is time-consuming.

The traditional EPLL based method utilizes constant regularization to smooth image. However, images often contain different image contents such as flat region, edge, textures and many tiny details. Therefore, global regularization parameter limits the performance of image denoising method. To improve the denoised results of the original EPLL based image denoising method, it is natural to explore adaptive regularization parameter estimation that can be able to set different parameter value to different image structure. Considering that the GMM can represent various data distributions, we will use its components to propose adaptive regularization parameter selection method. In fact, with the help of GMM, we can gain image clusters information. With the observation that in cluster every pixel shares the same content, it is natural to assign each cluster a parameter rather than each pixel. Therefore, in this paper we attempt to use the simple and effective lagrange multiplier to select regularization parameter for every cluster. In addition, to overcome the inherent drawback of lagrange multiplier we also introduce the local entropy into our estimation. It should be mentioned that our regularization parameter varies with cluster rather than local region.

This paper is organized as follows. In Section 2, we briefly review the Expected Patch Log Likelihood based image denoising method. Our proposed method with adaptive regularization parameters is presented in detail in Section 3. In Section 4, we demonstrate the experimental results. Section 5 concludes this paper.

2 Expected patch log likelihood method

Image u with N pixels can be divided into N overlapped image patches. Let \( {u}_i\in {\mathfrak{R}}^L \) be the vectorized version of an image patch of size \( \sqrt{D}\times \sqrt{D} \), obtained by ui = Piu, where Pi denotes an operator for extracting image patch ui from image u at position i. Given that there exist K mixture components and images patches are independent of each other, the density function of the GMM on ui is written as:

$$ p\left(\mathrm{u}\right)=\sum \limits_{\mathrm{j}=1}^{\mathrm{K}}{\uppi}_{\mathrm{j}}\mathrm{N}\left({\mathrm{u}}_{\mathrm{i}}|{\upmu}_{\mathrm{j}},{\Sigma}_{\mathrm{j}}\right) $$
(1)

Where πj is the mixing coefficient, μj and Σj are the mean and covariance matrix respectively, and N(ui| μj, Σj) is the Gaussian distribution defined as follows [9]:

$$ \mathrm{N}\left({\mathrm{u}}_{\mathrm{i}}|{\upmu}_{\mathrm{j}},{\Sigma}_{\mathrm{j}}\right)=\frac{1}{{\left(2\uppi \right)}^{\mathrm{D}/2}{\left|{\Sigma}_{\mathrm{j}}\right|}^{1/2}}\exp \left[-\frac{1}{2}{\left({\mathrm{u}}_{\mathrm{i}}-{\upmu}_{\mathrm{j}}\right)}^{\mathrm{T}}{\Sigma}_{\mathrm{j}}^{-1}\left({\mathrm{u}}_{\mathrm{i}}-{\upmu}_{\mathrm{j}}\right)\right] $$
(2)

Then, a prior called the EPLL for image u can be written as:

$$ \mathrm{EPLL}\left(\mathrm{u}\right)=\log \mathrm{p}\left(\mathrm{u}\right) $$
(3)
$$ \mathrm{p}\left(\mathrm{u}\right)=\prod \limits_{\mathrm{i}=1}^{\mathrm{N}}\sum \limits_{\mathrm{j}=1}^{\mathrm{K}}{\uppi}_{\mathrm{j}}\mathrm{N}\left({\mathrm{u}}_{\mathrm{i}}|{\upmu}_{\mathrm{j}},{\Sigma}_{\mathrm{j}}\right) $$
(4)

The EPLL based image denoising model is written as:

$$ \underset{\mathrm{u}}{\min}\left\{\frac{\uplambda}{2}{\left|\left|\mathrm{u}-{\mathrm{u}}^0\right|\right|}^2-\sum \limits_{\mathrm{i}=1}^{\mathrm{N}}\log \mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}\right)\right\} $$
(5)

where u0 is the original clean image, λ is the regularization parameter. Commonly, λ can be calculated by λ = D/σ2 where σ2 denotes the noise level. However, the σ2 is unknown in fact. Equation (5) can be solved by the Half Quadratic Splitting algorithm [31, 33].

3 Proposed method with adaptive parameters

Let {O1, …, OK} denote a partition of image using the GMM, and λ1, …, λK denote the regularization parameters of the above-mentioned clusters. We assume that the regularization parameters satisfy local constraints as follows:

$$ \frac{1}{\left|{\mathrm{O}}_j\right|}{\sum}_{i\in {\mathrm{O}}_j}{\left({\mathrm{u}}_i-{\mathrm{u}}_i^0\right)}^2={\upsigma}^2,\forall \mathrm{i}=1,2,\dots, \mathrm{K} $$
(6)

K denotes the number of clusters and |Oj| is the number of image paches in j − th class. With Eq. (6), the EPLL method can be written in a cluster-constraint way as follows:

$$ \underset{\mathrm{u}}{\max}\sum \limits_{\mathrm{i}}\log \mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}\right)\ \mathrm{s}.\mathrm{t}.{\sum}_{i\in {\mathrm{O}}_j}{\left({\mathrm{u}}_i-{\mathrm{u}}_i^0\right)}^2={\upsigma}^2\left|{\mathrm{O}}_{\mathrm{i}}\right| $$
(7)

Then Eq. (7) can be solved by the following unconstrained problem:

$$ \underset{\mathrm{u}}{\min}\left\{\frac{\uplambda \left(\mathrm{x}\right)}{2}{\left|\left|\mathrm{u}-{\mathrm{u}}_0\right|\right|}^2-\sum \limits_{\mathrm{i}}\log \mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}\right)\right\} $$
(8)

where \( \lambda (x)=\sum \limits_{j=1}^K{\lambda}_j\chi {O}_j \), χOj is the membership function of Oj.

$$ {\lambda}_j=1-\mathit{\exp}\left(-\frac{E_j}{\tau}\right) $$
(9)

where τ adjusts the decay of the exponential expression and Ej is a cluster-extended entropy, that is the entropy of cluster, which is calculated by

$$ {E}_j=-\sum \limits_{m=1}^{S_j-1}{p}_m^j{logp}_m^j $$
(10)

where \( {p}_m^j={W}_m/\left|{\mathrm{O}}_j\right| \) denotes the probability of the m-th gray level, Sj is the maximum gray level in Oj, and Wm is the number of pixels with the i-th gray level in cluster Oj. In this paper we proposed is an extension of GMM model, where the regularization parameters (λ1, …, λK) takes different values for different image contents that is the clusters. By using the Half Quadratic Splitting technique, the Eq. (10) can be equivalently transformed into the following function as:

$$ \underset{\mathrm{u},\left\{{\mathrm{z}}_{\mathrm{i}}\right\}}{\min}\left\{\frac{\uplambda \left(\mathrm{x}\right)}{2}{\left|\left|\mathrm{u}-{\mathrm{u}}_0\right|\right|}^2+\sum \limits_{\mathrm{i}}\left\{\frac{\upbeta}{2}\left({\left|\left|{\mathrm{P}}_{\mathrm{i}}\mathrm{u}-{z}_{\mathrm{i}}\right|\right|}^2\right)-\log \mathrm{P}\left({z}_{\mathrm{i}}\right)\right\}\right\} $$
(11)

For solving (11), at first, we choose the most likely Gaussian mixing weight jmax for each patch Piu.

$$ {\displaystyle \begin{array}{c}{\mathrm{j}}_{\mathrm{max}}=\underset{\mathrm{j}}{\max}\mathrm{p}\left(\mathrm{j}|{\mathrm{P}}_{\mathrm{i}}\mathrm{u}\right)\\ {}=\underset{\mathrm{j}}{\max}\mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}|\mathrm{j}\right)\mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}\right)\\ {}=\underset{\mathrm{j}}{\max}\left\{\log \mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}|\mathrm{j}\right)+\log \mathrm{p}\left({\mathrm{P}}_{\mathrm{i}}\mathrm{u}\right)\right\}\end{array}} $$
(12)

Then Eq. (11) is minimized by alternatively updating zi and u.

For a fixed un, updating zi is equivalent to solve the local MAP-estimation problem as follows:

$$ \underset{z_i}{\min}\left\{\ \frac{\beta }{2}{\left\Vert {P}_iu-{z}_i\right\Vert}^2-\sum \limits_i\log p\left({z}_i\right)\right\} $$
(13)

In fact the Wiener filter is:

$$ {z}_i^{n+1}={\left({\Sigma}_{j_{max}}+\frac{1}{\beta }I\right)}^{-1}\cdot \left({P}_i{u}^n{\Sigma}_{j_{max}}+\frac{1}{\beta }{\mu}_{j_{max}}I\right) $$
(14)

Where I is the identity matrix. For a fixed zi , an Euler-Lagrange formula can be obtained as follows:

$$ \lambda (x)\left(u-{u}_0\right)+\sum \limits_i\beta {P}_i^T\left({P}_iu-{z}_i\right)=0 $$
(15)

Then we have:

$$ \frac{\partial u}{\partial t}=\lambda (x)\left(u-{u}_0\right)-\sum \limits_i\beta {P}_i^T\left({P}_iu-{z}_i\right)=0 $$
(16)

Equation (16) can be further solved by the gradient descent algorithm and updating u as follows:

$$ {u}^{n+1}={u}^n+\varDelta t\left[\lambda (x)\left({u}_0-{u}^n\right)-\sum \limits_i\beta {P}_i^T\left({P}_i{u}^n-{z}_i^n\right)\right] $$
(17)

where ∆t is the time step.

In summary, the algorithm of our proposed denoising method is implemented as follows:

  1. Step1.

    Input corrupted image u0, parameters β, ∆t and iterations stopping tolerance ε ;

  2. Step2.

    Choose the most likely Gaussian mixing weights for each patch;

  3. Step3.

    Initially, set the values of λj ≥ 0 to be small enough so that

$$ {\mathrm{Q}}_{oi}\left({u}^{\lambda}\right)=\frac{1}{\mid {O}_i\mid }{\left\Vert {u}^{\lambda }-{u}_0\right\Vert}^2>{\delta}^2,\forall i=1,\dots K $$
(18)
  1. Step4.

    For each λj, we alternatively update (14) and (17), until reach the asymptotic state uλ.

  2. Step5.

    For each i ∈ {1, …, r}, recomputed λi = max(λi + ρ(Qoi(uλ − δ2), 0)) (with ρ > 0 small enough)

  3. Step6.

    Iterate steps 4-5 until the satisfying stopping criterion.

4 Experimental results

In experiments, we compare our proposed method with current popular mixture model based image denoising methods, including the original EPLL method [36], the Student’s-t Mixture Model based image denoising methods (SMM-EPLL) [33] and the EPLL based image denoising method using adaptive regularization parameters (EPLL-ARP) [31]. The GMM with 200 mixture components is learned from 2 × 106images patches which are sampled from the Berkeley Segmentation Database Benchmark (BSDS300) with their DC removed. Accordingly, in all experiments, the noisy images is generated by adding Gaussian noise with zero mean and standard variance σ2 = 25 into the test image with size of 481×321. That is the noise level of noisy image is 25. The parameters in our numerical experiments are as follows: the weighted coefficients β = 1/σ2 ∗ [1 4 8 16], the size of local entropy 3 × 3 and the image patch size D = 64.

Figure 1 shows the results of the four mixture model based image denoising methods. Figure 1a is the original clean Warcraft image in BSDS300 with No. 37073. Fig. 1b is the noisy image corrupted by Gaussian noise with noise level σ2 = 25. Figure 1c-d is the denoised images of traditional EPLL method, SMM-EPLL method, EPLL-ARP method and our proposed method. From the comparison of the denoised images, we can see that SMM-EPLL, EPLL-ARP and our method can yield better results than original EPLL method. By carefully comparing the above-mentioned method, we can see that the edges in images Fig. 1d and f are more clear. This means that SMM-EPLL and the herein proposed method can achieve a better tradeoff between noise removal and image detail preservation. The quantitative comparison of the four denoising method in terms of PSNR (Peak Signal to Noise Ratio) is displayed in Table 1.

Fig. 1
figure 1

Denoising results of the “Warcraft” image. a Original Warcraft image. b Noisy image with zero mean and variance σ2 = 25. c EPLL. d SMM-EPLL. e EPLL-ARP. f Our method

Full size image
Table 1 The PSNR results of different denoising methods (dB)
Full size table

Figure 2 compares the performance of the EPLL, the SMM-EPLL, the EPPL-ARP and our method on the Booby image with No. 103070 in Berkeley Database (BSDS300). We can see from the results that in the flat region SMM-EPLL and our method can generate visually satisfying denoised image. From Table 1, we can also observe that the PSNR values of SMM-EPLL and our method are higher that of EPLL and EPPL-ARP. Since the Student’s-t Mixture Model (SMM) is more robust than the Gaussian mixture mode, SMM based image denoising could obtain better result than EPLL. Moreover, SMM-EPLL also employ multi-scale technique to learn image prior, which can collect more priors on original image and further improve the performance of EPLL method. However, as shown in Table 2, SMM-EPLL is time consuming. By comparison, the time consumptions of the EPLL-ARP method and our method are comparatively low.

Fig. 2
figure 2

Denoising results of the “Booby” image. a Original image. b Noisy image with zero mean and variance σ2 = 25. c EPLL. d SMM-EPLL. e EPLL-ARP. f Our method

Full size image
Table 2 Computation times of image denoising methods (s)
Full size table

Figure 3 demonstrates the denoising results of the four EPLL method on the human-face image with No.302008 in Berkeley Database (BSDS300). Figure 3a is the original image. Figure 3b is the degraded image corrupted by Gaussian noise. Figure 3c–f displays the denoised image yielded by EPLL, SMM-EPLL, EPLL-ARP and our method. Through observing the results, our method can preserve fine texture details in image denoising. By magnifying the man’s eyes region and locating it in the lower right corner of the result image, we can see that small-scale textures of the men’s eyes can be preserved well.

Fig. 3
figure 3

Denoising results of the “Human-face” image. a Original image in BSDS300. b Noisy image with zero mean and variance σ2 = 25. c EPLL. d SMM-EPLL. e EPLL-ARP. f Our method

Full size image

Tables 1 and 2 respectively show the PSNR values and computation times of the four denoising method on the test image. As displayed in Table 1, SMM-EPLL performs the best in PSNR evaluation, for the reason that it jointly uses the SMM and multi-scale technique, which can learn more image priors and is robust to noise. Compared to EPLL-ARP method, our method obtain higher PSNR values. This is probably because that EPLL-ARP utilizes image gradient to select regularization. It is well-known that image gradient is sensitive to noise, therefore limiting the performance of EPLL-ARP. Although the PSNR value of SMM-EPLL is slightly higher than that of our method, it’s time consumption is obviously higher than ours.

Figure 4 demonstrates the denoising performance of EPLL, SMM-EPLL, EPLL-ARP and our method on Tiger image with No. 160068 in BSDS300. Tiger image is rich in details. Therefore it can be used to evaluate the performance of denoising method in texture-preservation. From the results, we can see our method can achieve satisfying denoised image. Fine details in Fig. 4f are preserved well. The Performance of our method on test images with different noise level is shown in Table 3.

Fig. 4
figure 4

Denoising results of the “Tiger” image. a Original image in BSDS300. b Noisy image with zero mean and variance σ2 = 25. c EPLL. d SMM-EPLL. e EPLL-ARP. f Our method

Full size image
Table 3 Performance of our method on test images with different noise level (dB)
Full size table

5 Conclusions

Image prior plays an important role in image restoration task. The GMM is a powerful tool for learning image prior and has drawn much attention in image processing. In this paper, we present a new regularization parameter estimation method using Lagrange multiplier technique, which employs local entropy to adaptively determine the regularization parameter. Each component of GMM corresponds to one regularization parameter. In other words, the regularization parameters are adaptive to the clusters. Each cluster is assigned to a regularization parameter, which means that our method can adjust the smoothing extent according to the image content in image denoising.

The herein proposed method is compared to three current popular mixture model based image denoising methods such as the original EPLL [36], SMM-EPLL [33] and EPLL-ARP [31], on different kinds of images including piecewise smooth image, human face image and texture image. Experiment results show that our method performances well both in visual effect and quantitative evaluation. SMM-EPLL method and our method yield good result for piecewise smooth image. However, the time consumption of SMM-EPLL is the highest because of its sophisticated multi-scale technique. On human face image and texture images, our method can preserve small-scale texture comparatively well compared to the other EPLL method using adaptive regularization parameters, that is the EPLL-ARP. The usage of gradient information to estimate regularization parameter influences the robustness of EPLL-ARP method. By contrast, our method is cost-effective.