Abstract
Gaussian mixture model learning based image denoising as a kind of structured sparse representation method has received much attention in recent years. In this paper, for further enhancing the denoised performance, we attempt to incorporate the gradient fidelity term with the Gaussian mixture model learning based image denoising method to preserve more fine structures of images. Moreover, we construct an adaptive regularization parameter selection scheme by combing the image gradient with the local entropy of the image. Experiment results show that our proposed method performs an improvement both in visual effects and peak signal to noise values.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Digital image has been widely used in our daily life. However, during the acquiring process, it is inevitably corrupted by the degraded factors, mainly including the precision in measurements of sensors, motion blur, lens aberration. Therefore, in order to obtain the high quality images, there has been a growing attention in image denoising techniques.
Generally speaking, given a corrupted image u 0 = A u + ε, where A stands for a linear blurring operator, ε is the additive white Gaussian noise and u is the original image, restoring u from u 0 often is an ill-posed inverse problem. In the past decades, a great variety of image recovery methods for solving the inverse problems have been presented, such as the regularization methods based [8, 12, 27, 31], the image sparse representation based [3, 6, 15, 28], the mixture models learning based [19, 25, 32], and so on. The total variation (TV) regularization [18, 29] as a classical image restoration method, utilizes priors on certain distributions of image gradients to locally regularize image, which enables to obtain satisfying results on piecewise-smooth images like cartoon images. The sparse representation method for image restoration is based on the fact that image can be sparsely represented on a redundant dictionary adaptively learned, and has proven its validity in recovering a wide variety of images [17, 30], including the natural images, the medical images, the aerial images and the satellite images. However, the traditional sparse representation method assumes that the atoms in dictionary are independent of each other and the structure information between the atoms is not taken into consideration. So as to achieve more satisfying denoised results, at present, many people have devoted to the research on the correlation among the dictionary atoms and propose a number of structured sparse representation methods.
The mixture model learning based image denoising method employs a small number of mixture components to learn priors over image patches for image statistical modeling, which has the advantages of the well-understood mathematical behaviors and relatively low computational complexity. Especially, the Gaussian Mixture Model (GMM) [11, 14, 16, 26] based image restoration method has proven its effectiveness and achieved good results, which actually is a kind of structured sparse representation method. Compared with the dictionary learning, learning image priors using GMM is less time consumption and easy to implement. Presently, the Expected Patch Log Likelihood (EPLL) which restores an image by maximizing the expected patch log likelihood, as one of the most important image denoising methods based on Gaussian mixture image prior has achieved a satisfying result.
In addition, the gradient information of image has been successfully introduced into the TV model, the pixel-level MAP-estimation [2] and non-local sparse representation [4, 5, 23], for preserving the small-scale textures of image. Besides, image gradient is frequently employed in image quality assessment, on account of the fact that it has the advantage of acquiring more fine structures of images. Motivated by this, we incorporate the gradient fidelity term [13] with the GMM based image denoising method for further enhancing the image denoising performs during the noise removal.
Moreover, recently, regularization parameter selection has received much attention, which has a great effect on preserving more details of image when denoising. At present, numerous methods for regularization parameter selection have been put forward, including the discrepancy principle based [21], the residual image statistics (RIS) based [7], the L-curve and gradient based [24]. In this paper, we focus on the image gradient based regularization parameter selection problem for GMM based image denoising method. Unfortunately, the image gradient is sensitive to noise and can not acquire satisfying results when the image is corrupted seriously. By observing that the local entropy of image has a good robustness to noise, we attempt to construct a new adaptive regularization parameter using image gradient matching and the local entropy of image. Furthermore, we analyze the relationship between the regularization parameters of the data-fidelity term and gradient-fidelity term, and propose a novel regularization parameter scheme for GMM based image denoising method for preserving more small-scale textures and details of images.
This paper is organized as follows: Section 2 briefly reviews the GMM based image denoising method, Section 3 introduces our proposed method with adaptive regularization parameters. In Section 4 we show experimental results and Section 5 give a simple conclusion.
2 GMM based image restoration model
Image u including N pixels can be divided into N overlapped image patches. Let u i (i=1,2,...,N) denote an image patch with the size of\(\sqrt L\times \sqrt L\) obtained by u i = R i u, where L is the number of pixels in each image patch, R i denotes an operator for extracting image patch u i from image u at position i. Given that there exist K mixture components and the patch u i is independent of each other, then the density function of the GMM on an image patch u i can be given by:
where π j is the mixing weights, μ j and Σ j are the corresponding mean and covariance matrix, N(u i |μ j ,Σ j )is the Gaussian distribution, which is defined as:
Owing to the independence of each patch, the joint conditional density of the image u=(u 1,u 2,...,u N ) can be modeled as:
the log of likelihood function is written as:
where Θ={π j ,μ j ,Σ j } is the set of the model parameters.Then the optimization can be solved easily by maximizing the (4) with the help of the Expectation Maximization algorithm (EM).
Lately, image priors as driving force for image denoising have been popular, which plays an important role in image denoising. One such a popular prior is learned by GMM. From (1), learning the priors on a given patch set with the EPLL is defined as follws:
Given a corrupted image u 0, restoring u from u 0 with the EPLL is equivalent to maximize the posterior probability:
Then the image denoising model based on EPLL is derived as:
where λ is the regularization parameter and the (7) can be solved by the Half Quadratic Splitting algorithm [1, 9, 10, 20].
3 Proposed method with adaptive regularization parameter
Regularization parameter selection plays an important role in image restoration. As explained in [22], an adaptive regularization parameter for the fidelity term is effective to preserve fine structures of images. This is naturally owing to the parameter in fidelity terms is adaptive to the image information and will change its value automatically in different regions of the image. Following the above motivation, we construct a new adaptive parameter coupling local entropy of images and analyze the relationship between the regularization parameters of the data-fidelity term and gradient-fidelity term. In this paper, for the sake of acquiring more satisfying denoised results, a novel Gaussian mixture model based image denoising method with adaptive fidelity term is presented as follows:
where ∇ denotes the image gradient, G σ is a Gaussian filter operator, ∗ is the convolution symbol, the first term imposes a global constraint on the restored image; the second term is the gradient fidelity term which describes the similarities in gradient between the corrupted images and the restored ones for preserving more image details. λ and α are the weight coefficients and can be chosen as a function of the gradient and local entropy of corrupted image in the following form:
where k is a given constant, k 0 is a threshold value, g(|∇u|) is designed by:
where k 1 is also a threshold value, when |∇u| is large, \(g(|\nabla u|)\rightarrow 2\), when \(|\nabla u|\rightarrow 0 \), \(g(|\nabla u|)\rightarrow 2+k_{1}\), E(x,y) is the local entropy of image and is defined by:
where p i = n i /(M×N) is the probability of the i-th gray level, S is the maximum gray level in the neighborhood centered on the pixel (x,y), in practice, we generally set the size of the local neighborhood as 3×3 and n i is the number of pixels with the i-th gray level, M×N is the size of image and f(E(x,y)) is defined as follows:
where G is the maximum of image gradient norm, and the range region of f(E(x,y)) is [1,G+1], which alleviates the impact of image gradient on the regularization parameters due to the introduction of the local entropy of images and makes the adaptive parameters have a better robustness.
The regularization parameter λ(x,y) and α(x,y) vary with different regions of the image. λ(x,y) is set to be small in the smooth regions of image, while α(x,y) is large so as to remove much noise in the meanwhile guarantee the similarity between the restored image and the corrupted one. At the edge of image, α(x,y) is small but λ(x,y) is large, for the sake of preserving more edges of the image. Meanwhile, the given k can help balance the regularization parameters λ(x,y) and α(x,y), and make them have proper values in different regions of the image for preserving more details of image while smoothing noises. Consequently, an adaptive fidelity term is useful and effective for preserving more textures and details of image in the process of noise removing.
Here, we employ the Half Quadratic Splitting algorithm to solve (8). The (8) is equivalently transformed into the following function by introducing a set of auxiliary variables {z i } as follows:
where β is the penalty parameter which usually is set to be large enough to ensure that the solution of (14) is close to that of (8).
For solving (14), at first, we choose the most likely Gaussian mixing weight j m a x for each patch R i u:
Accordingly, the image patch has the mean \(\mu _{j_{max}}\) and covariance matrix \({\Sigma }_{j_{max}}\), then (14) is minimized by alternatively updating z i and u.
For a fixed u n, updating z i is equivalent to solve the local MAP-estimation problem as follows:
It is in fact a Wiener solution:
where I is the identity matrix.
For a fixed z i , using the Euler-Lagrange formula for (14) and the corresponding Euler equation of it is:
the corresponding PDE as:
It can be solved by the gradient descent algorithm and updating u as follows:
where Δt is the time step, u x x ,u y y ,u 0x x ,u 0y y are respectively the second-order partial derivative of u and u 0.
In summary, our suggested algorithm can be implemented as follows:
-
Step1.
Input corrupted image u 0 , model parameters β,Δt and iteration stop- ping tolerance ε, initialize regularization parameters λ,α ;
-
Step2.
Choose the most likely Gaussian mixing weights j m a x for each image patch z i with the model (15);
-
Step3.
Calculate \({z_{i}^{1}}\) using (17);
-
Step4.
Calculate u 1 using (20) with updated regularization parameters λ and α;
-
Step5.
Calculate \(z_{i}^{n+1}\) using (17);
-
Step6.
Pre-estimate image u n+1 using (20) with updated λ and α;
-
Step7.
Repeat Steps 5-6 until satisfying stopping criterion.
4 Implementation and experiment results
In our experiments, the GMM with 200 mixture components is learned from a set of 2×106 images patches sampled from the Berkeley Segmentation Database Benchmark (BSDS300) with their DC removed. Comparing our proposed method with the original EPLL and the EPLL with Xie adaptive λ and α in [22], the parameters in our proposed method are as follows:the image patch size \(\sqrt L=8\), the weighted coefficients \(\beta =\frac {1}{\sigma ^{2}}*[1 \quad 4 \quad 8 \quad 16]\), the noise standard variance σ=25 and the size of local entropy of the image is set as 3×3, the constant \( k_{0}=\frac {1}{\sigma ^{2}}\), k 1=7 and \( k=\frac {L}{2\sigma ^{2}}\).
Figure 1 demonstrates the denoised results of the original EPLL and our proposed method with adaptive λ,α on Test1 image (i.e,No.3096). The related quantitative comparison, in terms of peak signal to noise ratio (PSNR) and signal to noise ratio (SNR), are shown in Table 1. The Fig. 1a and b are respectively the original image and noisy image. The Fig. 1c shows that the denoised result obtained by the original EPLL, and we can see that some regions of the image are not smooth, in contrast, the Fig. 1d obtained by our method shows a better result. This is probably due to the fact that our proposed method incorporates the gradient fidelity term with the EPLL and the regularization parameters are adaptive to the image information, which can help preserve more details of image and make the degraded image smoother during the denoising procedure. In addition, as demonstrated in Table 1, the PSNR value and SNR value of our method is higher than the original EPLL.
In Fig. 2. we also compare our proposed method with the original EPLL on Test2 image (i.e,No.3063). The related quantitative comparison, in terms of PSNR and SNR are shown in Table 2. From the denoised results shown in Fig. 2c and d, we can see that our proposed method outperforms the original EPLL and make the image smoother. In addition, we enlarge the middle part of the denoised result and put it on the right of image. From the Fig. 2d, we can find that our proposed method can better preserve the edges and small-scale textures of the image. This is because our proposed method has adaptive regularization parameters and can help preserve more details of images. Therefore, by comparison, our proposed method can obtain visually satisfying results and performs better in PSNR and SNR.
Figure 3. demonstrates the denoised results of our proposed method and the comparison with the EPLL with fixed λ,α on Barbara image. The corresponding PSNR and SNR are shown in Table 3. We enlarge the right shoulder of denoised result and put it on the right of image. From the result in Fig. 3c we can see that some small-scale textures of the image are not clear, while the result of our proposed method in Fig. 3d preserve more textures. This is probably on account of the fact that the regularization parameters being a constant of the method in Fig. 3c, while the parameters of our proposed method vary with different regions of the image. They can change their values automatically according to the image information and help preserve more fine structures in image. Therefore, by comparing, our proposed adaptive method outperforms the EPLL with a fixed λ,α both in PSNR and SNR.
Figure 4.demonstrates the denoised results of the EPLL with Xie λ,α and our proposed with adaptive λ,α on the Test3 image (i.e,No.160068). The related PSNR and SNR are shown in Table 4. The Fig. 4a and b are respectively the original image and noisy image. The Fig. 4c shows that the denoised result obtained by the EPLL with Xie adaptive λ,α and the Fig. 4d shows that the denoised result of our proposed method. We construct a new adaptive regularization parameter with local entropy of image. Due to the robustness of local entropy to noise, our proposed method can not only remove noise but preserve the weak edges and fine details of image. Thus, we can see that our proposed method can obtain a more satisfying result.
5 Conclusions
In this paper, a novel GMM based image denoising method with gradient fidelity term has been proposed, which can help preserve more small-scale textures and details of images during the noise removal. The GMM is a powerful tool for learning image priors, that is easy to implement and requires a small amount of parameters to estimate. In addition, compared with the dictionary learning in sparse representation, GMM has the advantages of relatively low computational complexity and the well-understood mathematical behavior. Furthermore, for preserving more weak edges of images when denoising, we construct a new adaptive selection scheme for the regularization parameters by means of the local entropy of the image, which varies with different regions of the image and has a good robustness to noise. Experiments show that our proposed method shows a clear improvement in comparison to the original EPLL algorithm and EPLL with fixed regularization parameters method for image denoising both visually and quantitatively.
References
Beck A. (2015) On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition scheme. SIAM J Optim 25(1):185–209
Chantas G, Galatsanos N, Likas A (2005) Maximum a posteriori image restoration based on a new directional continuous edge image Prior. In: Image Processing, 2005. ICIP 2005. IEEE International Conference on. IEEE, 1, I-941-4
Dabov K, Foi A, Katkovnik V et al (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095
Dong W, Shi G, Li X (2013) Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process 22(2):700–711
Dong W, Zhang L, Shi G et al (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630
Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745
Gilboa G, Sochen N, Zeevi Y Y (2006) Variational denoising of partly textured images by spatially varying constraint. IEEE Trans Image Process 15(8):2281–2289
Hu H, Froment J (2012) Nonlocal total variation for image denoising. In: 2012 Symposium on Photonics and Optoelectronics (SOPO), 1–4
Jain P, Netrapalli P, Sanghavi S. (2013) Low-rank matrix completion using alternating minimization. In: Proceedings of the forty-fifth annual ACM symposium on Theory of computing, ACM, 665–674
Kaganovsky Y, Degirmenci S, Han S et al (2015) Alternating minimization algorithm with iteratively reweighted quadratic penalties for compressive transmission tomography. SPIE Medical Imaging. International Society for Optics and Photonics, 94130J-94130J-10
Layer T, Blaickner M et al (2015) PET image segmentation using a Gaussian mixture model and Markov random fields. EJNMMI Physics 2(1):1
Liu K, Tan J, Ai L (2016) Hybrid regularizers-based adaptive anisotropic diffusion for image denoising. SpringerPlus 5(1):1
Lixin Z, Deshen X (2008) Staircase effect alleviation by coupling gradient fidelity term. Image Vis Comput 26(8):1163–1170
Nguyen T M, Wu Q M J (2012) Gaussian mixture model based spatial neighborhood relationships for pixel labeling problem. IEEE Trans Syst Man Cybern B Cybern 42(1):193–202
Peleg T, Eldar Y C, Elad M (2012) Exploiting statistical dependencies in sparse representations for signal recovery. IEEE Trans Signal Process 60(5):2286–2303
Qin Y, Ma H, Chen J et al (2015) Gaussian mixture probability hypothesis density filter for multipath multitarget tracking in over-the-horizon radar. EURASIP Journal on Advances in Signal Processing 2015(1):1–18
Ren J, Liu J, Guo Z (2013) Context-aware sparse decomposition for image denoising and super-resolution. IEEE Trans Image Process 22(4):1456–1469
Tang L, Fang Z (2016) Edge and contrast preserving in total variation image denoising. EURASIP Journal on Advances in Signal Processing 2016(1):1
Van Den Oord A, Schrauwen B (2014) The student-t mixture as a natural image patch prior with application to image compression. J Mach Learn Res 15(1):2061–2086
Wang Y, Yang J, Yin W et al (2008) A new alternating minimization algorithm for total variation image reconstruction. SIAM J Imag Sci 1(3):248–272
Wen Y W, Chan R H (2012) Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Trans Image Process 21(4):1770–1781
Xie C C, Hu X L (2010) On a spatially varied gradient fidelity term in PDE based image denoising. 2010 3rd International Congress on Image and Signal Processing (CISP). IEEE, 2, 835–838
Yan R, Shao L, Liu Y (2013) Nonlocal hierarchical dictionary learning using wavelets for image denoising. IEEE Trans Image Process 22(12):4689–4698
Yuan Q, Zhang L, Shen H et al (2010) Adaptive multiple-frame image super-resolution based on U-curve. IEEE Trans Image Process 19(12):3157–3170
Yu G, Sapiro G, Mallat S (2012) Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity. IEEE Trans Image Process 21(5):2481–2499
Yuhui Z, Byeungwoo J, Danhua X, Wu QMJ, Hui Z (2015) Image segmentation by generalized hierarchical fuzzy c-means algorithm. J Intell Fuzzy Syst 28(2):961–973
Zeng Y H, Peng Z, Yang Y F (2016) A hybrid splitting method for smoothing Tikhonov regularization problem. Journal of Inequalities and Applications 2016(1):1–13
Zhang J, Zhao D, Gao W (2014) Group-based sparse representation for image restoration. IEEE Trans Image Process 23(8):3336–3351
Zheng Y, Jeon B, Zhang J, Chen Y (2015) Adaptively determining regularization parameters in non-local total variation regularization for image denoising. Electron Lett:144–145
Zheng Y, Zhang J, Wang S, Wang J, Chen Y (2012) An Improved Fast Nonlocal Means Filter Using Patch-oriented 2DPCA. International Journal of Hybrid Information Technology 5(3):33–40
Zuo Z, Zhang T, Lan X et al (2013) An adaptive non-local total variation blind deconvolution employing split Bregman iteration. Circuits, Systems, and Signal Processing 32(5):2407–2421
Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: 2011 IEEE International Conference on Computer Vision (ICCV). IEEE, 479–486
Acknowledgments
This work was supported in part by the NSFC(Grants 61402234 and 61402235) and the PAPD.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, J., Liu, J., Li, T. et al. Gaussian mixture model learning based image denoising method with adaptive regularization parameters. Multimed Tools Appl 76, 11471–11483 (2017). https://doi.org/10.1007/s11042-016-4214-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-4214-4