1 Introduction

In the processes of image acquisition, it is inevitably contaminated by noise. It is necessary to remove the noise and improve the image quality, which will guarantee good performance of subsequent image processing [4, 21, 26]. The purpose of an image denoising method is to preserve image details and meanwhile remove noise. The researchers have proposed many denosing methods, ranging from spatial methods to transform domain methods, such as average filter, wavelets analysis and total variation [7, 14].

The average filter was once an effective method for removing noise. It is based on the pixel in spatial domain and its denoised edges become blurry. Antoni proposed a nonlocal means filter to remove Gaussian noise in 2005 [1]. The method utilizes redundancy of natural image. However, it is difficult to extract the redundancy of natural image. At the same time, Garnett presented a new trilateral filter to remove noise in 2005 [8]. It designs a weight cost function to calculate the weights of neighbor pixels, which is used for filter. It can remove noise effectively and meanwhile preserve the image edges. Unfortunately, the denoised results become very bad with the increase of noise. A few years later, Brox presented an improved non-local method by constructing a cost function in 2008 [2]. However, the cost function may be no effect while the noise drastically affects the direction of the gradient vector. On this basis, Li proposed a new non-local means method based on grey theory in 2016 [12]. Their experiments demonstrate that the method can extract signal from noise and meanwhile suppress pseudo-Gibbs artifacts. Kim proposed a feedback framework for image denoising in [11]. The proposed framework is solved by using split Bregman method.

The wavelet transform is a multi-scale method. It is used in signal analysis and image processing. The wavelet coefficients can distinguish signal from noise by threshold method in wavelet domain. Actually, the soft threshold is one of the most popular denoising methods, which is proposed by David in [5]. They have proposed that the shrinkage rule is near-optimal and then suggested that the optimal threshold could be chosen by minimizing SURE. Moreover, recent studies have found that increasing the redundancy of the wavelet transform is beneficial to the denoising performances. So Luisier proposed a novel SURE method for denoising in [13] and achieved some good denoised results. Yao proposed a denoising method by using principal component dictionary [22]. Their experiments demonstrate that their method outperforms several existing denoising methods.

In the last decade, total variation is one of the popular methods for image denoising. Total variation method was firstly proposed by Rudin [16]. It is described as an energy minimization problem. The method can suppress noise effectively. Nevertheless, it is easy to generate staircasing effects. A number of schemes have been proposed to solve the staircasing effects. In 2000, You proposed a fourth-order partial differential equation for noise removal [23]. In the method, a cost function is proposed based on the Laplacian of the image intensity function. Then the minimization of the cost function is solved by the time evolution of the partial differential equation. Ren proposed a novel model by introducing high-dimensional non-local total variation in [15] and then Shahdoosti proposed a new hybrid denoising scheme using the total variation in [17]. The visual quality demonstrates that their schemes can provide sharper edges but more or less bring some staircasing effects. Recently, the fractional-order total variation has been proposed for image denoising [20, 24, 25]. But it is not easy to minimize the cost function because of non-differentiability. To overcome the problem, a method is proposed to solve the total variation by the associated Euler–Lagrange equations in [3]. However, the regularization parameter in the total variation model is decided by experience, which fails to achieve the optimal denoised results.

To improve the total variation denoising performance, we do some researches mainly in two aspects. Firstly, the mixed total variation model is proposed to reduce staircasing effects. Secondly, different from the existing methods, the parameter estimation in our method is based on the decision-making scheme. To be more specific, the regularization parameter is estimated by generalized cross-validation approach and the optimal threshold is estimated by the noise level. Hence our method can remove noise and meanwhile preserve edge. Moreover, it can reduce staircasing effects. The experiments demonstrate that our method can achieve better denoised results than the state-of-the-art denoising methods.

This paper is organized as follows. Section 2 proposes a mixed total variation image denoising model. The improved noise estimation method is detailed in Section 3 and the proposed denoising method based on decision-making scheme is explained in Section 4. Experimental results are shown in Section 5 and finally conclusions are presented in Section 6.

2 Mixed total variation denoising model

The noise model can be described as g = f + n. f denotes a sharp image, g denotes a noisy image and n denotes additive noise. Then, the least square approximation of sharp image is achieved by solving the following variation problem.

$$ \underset{u}{\operatorname{inf}}\left\{\underset{\varOmega }{\int }{\left|g-u\right|}^2 dx\right\} $$
(1)

where u represents a denoised image. The denoised image u is achieved by obtaining the infimum of the above integral. However, this solution is an ill problem owing to noise. Hence, it is necessary to introduce regularization such as Laplacian regularization and total variation regularization. By introducing total variation regularization, the denoising model is obtained and shown as

$$ \underset{u}{\min }J(u)=\frac{1}{2}{\left\Vert u-g\right\Vert}^2+\lambda \int {\left|\nabla u\right|}^2 dx $$
(2)

where \( \mid \nabla u\mid =\sqrt{{\left({D}_xu\right)}^2+{\left({D}_yu\right)}^2} \) represents the gradient at pixel point. The integral of |∇u|2 represents the total variation of image. The first term represents fidelity term and the second term represents regularization term. λ represents regularization parameter. In the next years, T. F. Chan proposed a novel total variation model, shown in (3).

$$ \underset{u}{\min }J(u)=\frac{1}{2}{\left\Vert u-g\right\Vert}^2+\lambda \int \mid \nabla u\mid dx $$
(3)

where the total variation is anisotropic because of 1-norm used as the regularization term. It can effectively remove noise and preserve edge. However, it is easy to generate staircasing effects. On the contrary, the total variation is isotropic if 2-norm is used as the regularization term, as shown in (2). It can remove noise effectively and unfortunately bring blurry edge. Hence, the mixed total variation denoising model is proposed to improve the denoising performance, which is shown as

$$ \underset{u}{\min }J(u)=\frac{1}{2}\left\Vert u-g\right\Vert +\lambda \int {\left|\nabla u\right|}^{\varphi \left(|\nabla u|\right)} dx $$
(4)

where J(u) is a cost function, φ(| ∇u| ) is a mixed total variation regularization. If the gradient of the image at some point is very big, the point can is considered as edge. On the contrary, the point is considered as smooth region. So the function φ is defined as

$$ \varphi \left(|\nabla u|\right)=\Big\{{\displaystyle \begin{array}{ll}1,& \mid \nabla u\mid \ge Th\\ {}2,& \mid \nabla u\mid < Th\end{array}} $$
(5)

where Th is threshold, which is chosen according to the noise level. Moreover, compared with 2-norm, 1-norm is suitable for describing the sparsity. So 1-norm is introduced into the fidelity term in this paper.

3 The improved noise estimation method

3.1 Introduction to noise estimation method based on wavelet

Since the threshold Th is chosen according to the noise level, it is important to estimate the noise level. Image can be decomposed using wavelet transform and its low-frequency part reserves the majority of information. At the same time, the majority of its high-frequency part is noise. So the noise levels can be estimated by high-frequency sub-band coefficients. The noise estimation method based on Wavelet Coefficient Median (WCM) is proposed by Donoho D L in [6] and it is a general and effective noise estimation method at present. The formula is shown as

$$ {\sigma}_{\mathrm{original}}=\frac{Median\left(|Y\left(i,j\right)|\right)}{0.6745} $$
(6)

where Y(i, j) denotes wavelet coefficients of HH sub-band. Median() denotes median of absolute value of Y(i, j). σoriginal denotes the estimated standard deviation of noise. Furthermore, the used wavelet is d4 wavelet.

3.2 Improved noise estimation method based on wavelet coefficient median

To observe the effect of WCM, we synthesize some noisy images with different standard deviation of noise, including 2, 4, 6, 8, 11, 14, 20, 26. The estimation results are shown in Table 1. It is observed that the results are all over-estimation and the estimated errors become small with increasing noise. Actually, all of the high-frequency sub-band coefficients are considered as noise in WCM. However, the high-frequency sub-band coefficients not only contain noise, but also contain some image details. So it is necessary to improve WCM.

Table 1 The estimation results of WCM
Full size table

As seen from Fig. 1, it is found that the estimated error has a negative power relationship with the estimated noise. Hence, the fitting form is defined as

$$ e=\frac{\sigma_{\mathrm{original}}-{\sigma}_{\mathrm{actual}}}{\sigma_{\mathrm{actual}}}=a\ast {\sigma}_{\mathrm{original}}^b $$
(7)

where σoriginal represents the estimated noise by WCM and σactual represents the actual noise. The parameters a and b in (7) are obtained by least square method.

$$ a=17.64\kern1.75em b=-2.331 $$
(8)
Fig. 1
figure 1

Relation of estimated error to estimated noise (X axes represents the estimated standard deviations of noise; Y axes represents the estimated error)

Full size image

Then the Improved noise estimation method based on Wavelet Coefficient Median (IWCM) is deduced by (6)~(8)

$$ {\sigma}_{\mathrm{improved}}={\sigma}_{\mathrm{actual}}=\frac{Median\left(|Y\left(i,j\right)|\right)}{0.6745\left(1+17.64\ast {\sigma}_{\mathrm{original}}^{-2.331}\right)} $$
(9)

3.3 The verification of IWCM

To verify the validity of IWCM, we test IWCM on the 512 × 512 airport, 512 × 384 sailboat, 512 × 384 parrot and 256 × 256 boat, which are added by some additive noise with different noise levels (σ: noise standard deviation). The noise levels are 2, 6, 10, 14 and 18. Then the noise levels are estimated by WCM and IWCM and shown in Tables 2. It is found that compared with WCM, the estimated errors of IWCM decrease greatly. So the noise levels will be estimated by IWCM in the following sections.

Table 2 The estimated results by WCM and IWCM
Full size table

4 The proposed denoising method based on decision-making scheme

4.1 The solution of total variation model

There exists a very popular method for minimizing the cost function. The method is called as alternating minimization method. It is found that the cost function in (4) is a convex function. Hence, after initialization of image, the cost function decreases when iteration number increases. However, derivation of 1-norm of matrices needs to be solved in the procedure of solution. Two auxiliary variables are employed to turn the cost function (4) into the new cost function (10), which is easier to solve.

$$ \underset{w,z,u}{\min }J\left(w,z,u\right)=\frac{1}{2}\left[\left\Vert z\right\Vert +\frac{\alpha }{2}{\left\Vert z-\left(u-g\right)\right\Vert}^2\right]+\lambda \sum \limits_{i=1}^{MN}\left[{\left\Vert {w}_i\right\Vert}^{\varphi }+\frac{\beta }{2}{\left\Vert {w}_i-{\nabla}_iu\right\Vert}^2\right]\kern0.5em \alpha, \beta \to \infty $$
(10)

where the parameters α, β → ∞, \( \underset{u}{\min }J(u) \) will lead to ‖z − (u − g)‖ → 0 and ‖wi − ∇iu‖ → 0. Namely formula (10) is equivalent to formula (4). Actually, formula (10) is minimized by solving wn + 1 = arg minwJ(w, un), zn + 1 = arg minwJ(zn, un) and solving un + 1 = arg minwJ(wn + 1, zn + 1, un). The cost function J(w, z, u) will be decreased when iteration number increases. And thus the alternating minimization scheme is summarized in the following algorithm.

In general, there may be no unique solution on the minimization of the cost function. It is necessary to impose the physical and natural condition on denoised image u so as to achieve a meaningful result. Similar to the ref. [10], a physical condition u ≥ 0 is imposed on the denoised image. Namely the denoised image should be nonnegative. In the next sub section, the minimizations of w, z, u need to be solved and the regularization parameter needs to be suitably decided, which are the most important in the procedure of our denoising method.

The update for w is achieved by solving wn + 1 = arg minwJ(w, un). However, there are only two values about w in the formula (5). So the optimization of w can be simplified into two cases.

When φ = 2, the solution is followed as

$$ {w}_{n+1}=\frac{\beta \nabla {u}_n}{\beta +2} $$
(11)

When φ = 1, the solution is followed as

$$ {w}_{n+1}=\max \left\{\left\Vert \nabla {u}_n\right\Vert -\frac{1}{\beta },0\right\}\frac{\nabla {u}_n}{\left\Vert \nabla {u}_n\right\Vert } $$
(12)

Similarly, the optimization of z is achieved and shown as

$$ {z}_{n+1}=\max \left\{\left\Vert {u}_n-g\right\Vert -\frac{1}{\alpha },0\right\}\frac{u_n-g}{\left\Vert {u}_n-g\right\Vert } $$
(13)

At last, the update for u is achieved by solving un + 1 = arg minwJ(wn + 1, zn + 1, un)

$$ \frac{\partial J}{\partial u}=\frac{\alpha }{2\lambda \beta}\left(u-g-z\right)+{\nabla}_{-x,-y}\left({\nabla}_{x,y}u-w\right)=0 $$
(14)

Then solving formula (14) in frequency domain, we achieved the update for u

$$ {u}_{n+1}={F}^{-1}\left[\frac{\alpha /\left(2\lambda \beta \right)\left(G+{Z}_{n+1}\right)+{(D)}^{\ast }{W}_{n+1}}{\alpha /\left(2\lambda \beta \right)+{(D)}^{\ast }D}\right] $$
(15)

where G, Z, D, W represents Fourier transform of g, z, ∇ , w. It will be proved that employing the update formulas (11)~(15) can improve denoising performance in Section 5.

figure a

4.2 Initialization of parameters

There are some parameters which need to be initialized in the proposed model. The initial denoised image u0 is selected as g as it is an effective approximation of sharp image. The stopping criterion ξ is set to be 10−4.

As summarized in sub section 4.1, when the auxiliary parameters α, β → ∞, the minimization of formula (10) is equivalent to the minimization of the formula (4). However, the convergence rate decreases as α and β increase. Considering the equivalence of cost function and convergence of algorithm, the auxiliary parameters are set to be a geometric sequence. The initial value and final value of the geometric sequence are set to be 1 and 107 respectively. After every iteration of u, these auxiliary parameters are modified as α = 3α, β = 3β.

Moreover, λ is a regularization parameter which is used for balance. In previous articles, it is usually chosen by experience. But in this paper, it is estimated by generalized cross-validation approach.

Generalized cross-validation is an approach that estimates a parameter directly with no prior knowledge. The core of concept is to extract some testing data out of the observation data and employ the remaining data to predict the testing data. The more accurately the parameter is estimated, the smaller the error of prediction is. So the parameter can be estimated by achieving the minimum of the error. To estimate the parameter more easily, the cost function is rewritten into new form.

$$ {\displaystyle \begin{array}{c}\underset{u}{\min }J(u)=\frac{\alpha }{2}{\left\Vert z-\left(u-g\right)\right\Vert}^2+\lambda \beta {\left\Vert w-\nabla u\right\Vert}^2\iff \\ {}\underset{u}{\min }J(u)=\frac{1}{2}{\left\Vert z-\left(u-g\right)\right\Vert}^2+\lambda {\left\Vert w-\nabla u\right\Vert}^2=\frac{1}{2}{\left\Vert u-\left(z+g\right)\right\Vert}^2+\lambda {\left\Vert \nabla u-w\right\Vert}^2\kern1em \mathrm{as}\ \alpha =\beta \end{array}} $$
(16)

The estimated regularization parameter is achieved by using the conclusion in Reference [9], shown as

$$ f\left(\lambda \right)={n}^2\sum \limits_{i=1}^{n^2}{\left(\frac{\mid {\left(w-\nabla \left(z+g\right)\right)}_i\mid }{{\left|{\left(\nabla \right)}_i\right|}^2+1/\lambda}\right)}^2/{\left(\sum \limits_{i=1}^{n^2}\frac{1}{{\left|{\left(\nabla \right)}_i\right|}^2+1/\lambda}\right)}^2 $$
(17)

where λ is computed by minimizing f (λ). Some numerical methods are proposed and employed to achieve the optimal λ. In this paper, we employ parabolic interpolation and golden section search to achieve the optimal λ [19].

5 Experimental results

Now the experiments are presented to demonstrate the performance of our method. Moreover, we compare our method with five denoising methods, including SURE in [13], RPTV in [17], NLM in [12], PCDPG in [22] and FFAF in [11].

In the following section, we employ the above five methods and our method (DBTV) to some synthetic noisy images and the test noisy images on TID2008. These original images are all standard images, such as Boat, Peppers, Bridge and Lena, shown in Fig. 2. One of these noisy images is shown in Fig. 3. To comprehensively evaluate the denoised images, PSNR and SSIM are used as criterions for evaluation.

Fig. 2
figure 2

The standard images

Full size image
Fig. 3
figure 3

The noisy image with the standard deviation of noise equaling to 8 (PSNR = 22.30)

Full size image

5.1 The choice of optimal threshold

We employ our method to the noisy image. The standard deviation of noise (σ) is 20. The selective threshold ranges from 0 to 100 and the interval of two adjacent thresholds is 1. PSNR of denoised images with different threshold are shown in Fig. 4.

Fig. 4
figure 4

The relation of PSNR of denoised image and Threshold

Full size image

As shown in Fig. 4, the peak of PSNR appears when the threshold is 24.1. Moreover, the noise levels (σ) is estimated by IWCM proposed in section 3.2. The estimated value is 20.26. Similarly, four images in Fig. 2 are added by different noise. The noise levels (σ) are 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30. Then the optimal threshold and the estimated noise (σ) are obtained and displayed in Fig. 5. It is found that the optimal threshold has a linear relationship with the estimated noise. Hence, the fitting form is defined as

$$ Th={a}_i\sigma +{b}_i $$
(18)

where i represents the i-th images and the corresponding curve are respectively displayed in Fig. 5(a)~(d). The parameters in (18) are achieved by least square method, shown in (19) and (20)

$$ {a}_1=1.108,\kern0.75em {a}_2=1.064,\kern0.75em {a}_3=1.258,\kern0.75em {a}_4=1.196 $$
(19)
$$ {b}_1=0.37,\kern0.75em {b}_2=-0.32,\kern0.75em {b}_3=0.06,\kern0.75em {b}_4=0.14 $$
(20)
Fig. 5
figure 5

The relation of the optimal Threshold and the estimated noise

Full size image

It is noted that these coefficients ai and bi are respectively almost equal. So we calculate the mean as the finally results.

$$ \overline{a}=\left({a}_1+{a}_2+{a}_3+{a}_4\right)/4=1.157\kern0.5em \overline{b}=\left({b}_1+{b}_2+{b}_3+{b}_4\right)/4=0.06\approx 0 $$
(21)

The optimal threshold is obtained by the following formula,

$$ Th=1.157{\sigma}^{\prime } $$
(22)

5.2 Comparison with the other denoising methods

To validate the performance of our method, we compare the proposed method with five denoising methods and show the results for the visual comparison.

In this sub section, we use some grayscale images for experiments, including Boat, Bridge, Lena, Peppers. To corrupt them, additive noise with different noise levels equal to 10, 20, 30 and 40 have been added to the sharp images. The denoised results of Boat with the noise level σ = 20 are shown in Fig. 6. Similarly, the denoised results of Peppers with σ = 40 are shown in Fig. 7.

Fig. 6
figure 6

The denoised results of Boat. a original image; b the noisy image with σ = 20; c the magnified image cropped from (a); d the magnified image cropped from (b); e the denoised image by SURE; f the denoised image by RPTV; g the denoised image by NLM; h the denoised image by PCDPG; i the denoised image by FFAF; j the denoised image by DBTV (our method)

Full size image
Fig. 7
figure 7

The denoised results of Peppers. a original image; b the noisy image with σ = 40; c the magnified image cropped from (a); d the magnified image cropped from (b); e the denoised image by SURE; f the denoised image by RPTV; g the denoised image by NLM; h the denoised image by PCDPG; i the denoised image by FFAF; j the denoised image by DBTV (our method)

Full size image

As seen from Figs. 6 and 7, it is found that the denoised images of our method have less noise and more clearly edges. We also find that compared with the other five methods, the denoised images of our method (Figs. 6j and 7j) have fewer staircasing effects and better vision effect, which demonstrates that our method outperforms five methods in terms of visual quality.

In addition, PSNR and SSIM of the denoised results from various algorithms are shown in Table 3, in which the best results are marked in boldface. As seen from Table 3, our method has excellent evaluation values for these noisy images with different noise level, which has shown that our method is the best method among these methods in terms of objective evaluation.

Table 3 The PSNR and SSIM of denoising methods
Full size table

5.3 The denoising results on TID2008

In this sub section, a large set of experiments are reported to validate the performance of our method. We test 25 noisy images on TID2008 [18], which is a publicly available image databases. The database includes many noisy images and benefit for image denoising, image analysis and image assessment. The part of the results are shown in Fig. 8 and all of the evaluation results are shown in Fig. 9.

Fig. 8
figure 8

The denoised results a the noisy images; b the magnified images cropped from column (a); c the denoised results by SURE; d the denoised results by RPTV; e the denoised results by NLM; f the denoised results by PCDPG; g the denoised results by FFAF; h the denoised results by DBTV

Full size image
Fig. 9
figure 9

The evaluation values of the denoised results of Fig. 8: a PSNR; b SSIM

Full size image

As seen from Fig. 9, the denoised results of our method (DBTV) have better vision effect. And meanwhile, it is found that the majority of PSNR and SSIM of our method is the biggest among all evaluation results. The average of PSNR and SSIM of these methods is shown in Table 4. Apparently, compared with the other methods, the proposed method can increase PSNR by 0.5 dB~1.5 dB and SSIM by 0.02~0.05. In a word, it is shown that our method is the best method among these methods in terms of visual quality and objective evaluation.

Table 4 The average of PSNR and SSIM in Fig. 9 (The best result is marked in bold)
Full size table

6 Discussion and conclusion

Image denoising is an important preprocessing step for image processing. In this paper, a denosing method based on mixed total variation regularization with decision-making scheme is proposed to improve the denoising performance. It firstly employs the proposed mixed total variation model to reduce staircasing effects. Then based on the decision-making scheme, the optimal threshold is estimated by the noise level and the regularization parameter is estimated by the generalized cross-validation approach. The comparison of the denoised results demonstrates the efficiency of our method which gives the best PSNR and SSIM. The visual quality of our results is moreover characterized by fewer staircasing effects than the other five methods.

In future work, we will combine the multi-scale analysis with the proposed mixed total variation method to further improve the denoising performance. Meanwhile, we will employ the method into image restoration to improve restoration results.