Wavelet Frame Based Image Restoration via Combined Sparsity and Nonlocal Prior of Coefficients

Abstract

Owing to the good ability of sparsely approximating piece-wise smooth functions like images, the (tight) wavelet frame has been widely investigated and applied for image restoration and other image processing problems over the past few years. Most of the variational models based on wavelet frame proposed in the past utilize the $l_{1}$ norm of frame coefficients as a sparsity prior. Very recently, the variational model which penalizes the $l_{0}$ “norm” of frame coefficients was proposed for image restoration, and proved to outperform the commonly used $l_{1}$ minimization methods in the quality of restored images. Though the $l_{0}$ “norm” has the ability of preserving sharp edges and smooth regions, textures and small details may be mistakenly removed at the same time. Therefore, we introduce a $l_0-l_2$ regularization model which contains a nonlocal prior of frame coefficients to avoid this issue in this paper. Meanwhile, a narrow-band technique is introduced to further improve the computational efficiency of the proposed algorithm. Numerical experiments demonstrate that the propose algorithm is superior to the recently proposed algorithm for $l_{0}$ “norm” minimization in both iterative time and recovery quality.

1 Introduction

Image restoration is one of the most important research subjects in many areas of image processing and computer vision. Its goal is to enhance the quality of a given image that is contaminated in various ways during the imaging process, and enable us to observe crucial details residing in the image. Restoration task can often be formulated as an ill-posed inverse problem:

$$\begin{aligned} f = K u + \epsilon \end{aligned}$$

(1)

where $u$ and $f$ denote the original image and the observed image respectively, $\epsilon$ is the Gaussian white noise with variance $\sigma ^{2}$, and $K$ is some linear operator, such as a convolution operator for image deconvolution problems. The goal of image restoration is to recover a sharp image from the degraded image $f$. due to the serious ill-condition of the operator $K$, the results of using direct inverse filtering are poor due to severe noise amplification. Therefore, proper regularization techniques, which contain variational models [1–3] and wavelet frame based approaches [4–8], should be adopted to suppress the impact of noise and obtain a satisfactory solution.

One of the most popular variational models is the Rudin–Osher–Fatemi (ROF) model [1] which penalizes the total variation (TV) norm of the image. The ROF model is especially effective for restoring images that are piece-wise constant, such as the binary image. However, it is very known that the TV regularizer suffers from the so-called stair-case effect [9], which may produce undesirable blocky images. In order to overcome the shortcoming, various variational models were further proposed, cf. [10–14]. On the other hand, many of the current PDE based methods with application to image denoising, deblurring and so on also utilize TV regularizer or its variants due to the edge-preserving ability. For more details refer to [2, 15].

In recent years, the sparsity-inducing prior based on wavelet frames has been playing a very important role in the development of effective image restoration models. The basic idea behind it is that the interest image can be sparsely represented by properly designed wavelet frames, that is, most of the important information including in the image can be kept by few wavelet coefficients. Therefore, one commonly used regularization term for the wavelet frame based models is the $l_{1}$-norm of frame coefficients. The results in literature [5, 16, 17] demonstrate that wavelet frame based approaches outperform the earlier variational models such as the ROF model, due to the multi-resolution structure and redundancy of wavelet frames. In fact, the recent research in [17] established the close connection between the variational model and one of wavelet frame based approaches, called the analysis based approach. It has been proved that the analysis based approach with different regularization parameters and tight frame systems can be seen as different discrete approximations of general variational models. Such connection also explains the advantage of the wavelet frame based approaches over the past variational models, that is, they can adaptively choose proper differential operators in different regions of a given image according to the order of the singularity of the underlying solutions. The connection of PDE based methods, variational models and wavelet frame based approaches was further investigated in [18, 19].

Most of the wavelet frame based models, including synthesis based approach, analysis based approach and balanced approach [4–7], utilize the $l_{1}$-norm of frame coefficients as the regularization term. Theories of compressed sensing developed by Candes and Donoho [20–22] guarantee that the $l_{1}$ minimization problem can produce a reliable recovery of the unknown image as long as proper conditions for the linear operator $K$ and the wavelet transform matrix are satisfied, and the transform coefficients of the unknown image are sparse enough. However, for practical problems such as image deblurring and magnetic resonance image reconstruction, these conditions are not necessarily satisfied. Therefore, these $l_{1}$-norm based models cannot always produce high quality recovery results for these applications. In order to place more emphasis on the sparsity of wavelet frame coefficients and produce recovery image with more sharp edges, $l_{p}$ quasi-norm with $0<p<1$ was used [23, 24] for the description of the sparsity instead. However, the proximal operator of the $l_{p}$ quasi-norm does not have a closed solution in contrast to the $l_{1}$-norm. Recently, Zhang et al. [25] proposed to penalize the $l_{0}$ “norm” of wavelet frame coefficients. In order to solve the analysis based approach with $l_{0}$ minimization, an algorithm called the penalty decomposition (PD) method, based on the earlier works [26], was proposed to solve this problem. However, the computational cost of the PD method is a little high due to the non-convexity of the $l_{0}$ “norm”. In the recent paper [27], a more efficient algorithm, called mean doubly augmented lagrangian (MDAL) method, was proposed for this $l_{0}$ minimization problem. Numerical experiments in [27] demonstrate that the analysis based approach based on $l_{0}$ minimization can generate recovery images with higher quality than the counterpart based on $l_{1}$ minimization, and the MDAL method is more efficient than the PD method.

Though the analysis based models with $l_{0}$ “norm” is able to preserve the sharpness of the edges and features, and retain the smoothness of the recovered images, the textures and tiny details which are included in the small frame coefficients may be mistakenly removed by the hard thresholding operator related to the $l_{0}$ “norm”. One difficult issue is how to preserve these coefficients that represent the important textures and detail information. Note that this drawback of frame based approaches is due to the fact that the elements of the wavelet frames discussed above are all locally supported in the spatial domain, and the wavelet transform is just equal to finite-difference operators defined in the neighborhood. In other words, the sparse representation of images under certain wavelet frames only reveals the sparse nature of local variations of image intensities. Therefore, the wavelet frame based models discussed above are called local approaches. They work very well on strong edges and smooth regions of images, but are not suitable for the description of textures regions.

Recently, a class of nonlocal image restoration techniques have attracted increasingly more attention. It is started with the nonlocal means (NLM) filter proposed by Buades et al. [28], which makes full use of the similarity of image pixels that may be far away in spatial domain. In this method, the similarity of pixels is not computed from the pixels themselves but from their neighboring patches. The NLM filter is suitable for exploring the redundancy information in images and shown to be very efficient for textures. Such method has been extended to solve various problems of image processing, see [29–31] for instance. Very recently, the nonlocal idea is combined with patch-dictionary methods, and generate various novel image restoration models [32–36, 45, 46]. These nonlocal methods were among the current state-of-the-art approaches in the quality of recovered images. However, the corresponding computational burden is too high due to the fact that a large number of image patches need to be clustered and sparsely coded during the iteration. Very recently, data-driven local or nonlocal wavelet frames [37, 38] were also developed for image restoration. However, the computational cost is even higher than some patch-dictionary methods. The experiments in literature [38] demonstrate that the MATLAB implementation of their algorithm takes about 9 min for an image with size $256\times 256$.

To balance the implementation time and the recovery quality, we propose a wavelet frame based image restoration model with a $l_0-l_2$ regularizer. In the proposed model, a nonlocal prior of the frame coefficients, in the terms of $l_2$ norm, is introduced in the variational model. It plays an important role of estimating the coefficients that contain the textures and finer details of images, which may be mistakenly removed by only utilizing the $l_{0}$ regularizer. The numerical examples below demonstrate that the proposed method can not only preserve the edges and smooth regions, but also recover the textures, which verify the efficiency of the added $l_2$ regularizer. Besides, from the experiments below we observe that the iterative algorithm for the proposed model can achieve the same accuracy with much less iteration number compared with the MDAL algorithm for $l_{0}$ minimization problem. To further accelerate the proposed algorithm, a technique called “narrow-band” is introduced to reduce the computation burden related to the estimation of the nonlocal prior. In the following experiments we find that the computational time of the iterative process of the proposed algorithm is even less than that of the MDAL algorithm for $l_{0}$ minimization [27].

The rest of this paper is organized as follows. In the next section, we first review some basic notions of wavelet tight frames. Then we further discuss the current wavelet frame based image restoration models utilizing the $l_1$ or $l_0$ regularization term. In Sect. 3, we introduce a nonlocal prior of frame coefficients in the form of the $l_2$ norm into the recently proposed model based on the $l_0$ minimization problem, and propose a new restoration model utilizing the $l_0-l_2$ regularizer. In order to reduce the computational cost of the nonlocal estimation, a new strategy called the narrow-band technique is also proposed. In Sect. 4, we provide comprehensive tests of the performance of the proposed algorithm for image deblurring; and compare it with the preconditioned Bregmanized operator splitting (PBOS) algorithm for solving the nonlocal total variation (TV) model [30], the MDAL method for solving the $l_0$ minimization problem [27], and the Iterative Decoupled Deblurring BM3D (IDD-BM3D) algorithm [46]. The conclusion is given in Sect. 5.

2 Wavelet Frames and Previous Works

In this section, we briefly introduce some preliminaries of wavelet tight frames, and then review some of the current frame based image restoration models.

Wavelet tight frames are the widely used frames in image processing. One wavelet frame for $L_{2}({\mathbb {R}})$ is a system generated by the shifts and dilations of a finite set of generators $\varPsi =\{\varPsi _{1}, \ldots , \varPsi _{n}\}$:

$$\begin{aligned} X(\varPsi )=\left\{ \varPsi _{q,j,k}, ~~1\le q\le n, j\in {\mathbb {Z}}, k\in {\mathbb {Z}}\right\} \end{aligned}$$

(2)

where $\varPsi _{q,j,k}=2^{j/2}\varPsi _{q}(2^{j}\cdot -k)$. Such set $X(\varPsi )$ is called a tight frame of $L_{2}({\mathbb {R}})$ if

$$\begin{aligned} f=\sum _{\psi \in \varPsi }\langle f, \psi \rangle \psi , ~~~\forall f\in L_{2}({\mathbb {R}}). \end{aligned}$$

(3)

The construction of framelets can be obtained by the unitary extension principle (UEP). For more details refer to [6, 39, 40]. One widely used framelet in image recovery is the linear B-spline framelet, which has two generators and the associated masks $\{h_{0}, h_{1}, h_{2}\}$ are

$$\begin{aligned} h_{0}=\frac{1}{4}\left[ 1,2,1\right] ; ~~h_{1}=\frac{\sqrt{2}}{4} \left[ 1,0,-1\right] ; ~~h_{2}=\frac{1}{4}\left[ -1,2,-1\right] . \end{aligned}$$

(4)

Given the 1-dimensional wavelet tight frame, the framelets for $L_{2}({\mathbb {R}}^{2})$ can be easily constructed by taking tensor products of 1-dimensional framelets.

In the discrete setting, assume that $W\in {\mathbb {R}}^{M\times N}$ ($M>N$) denotes the transform matrix of framelet decomposition. Then by the unitary extension principle we have $W^{T}W=I$. The matrix $W$ is called the analysis (decomposition) operator, and its transpose $W^T$ is called the synthesis (reconstruction) operator. The $L$-level framelet decomposition of $u$ will be further denoted as

$$\begin{aligned} W u=\left\{ W_{l,j}u: ~~0\le l\le L-1, j\in {\fancyscript{I}}\right\} \end{aligned}$$

(5)

where ${\fancyscript{I}}$ denotes the index set of the framelet bands, and $W_{l,j}u$ is the wavelet frame coefficients of $u$ in band $j$ at level $l$. The frame coefficients $W_{l,j}u$ can be constructed from the masks associated with the framelets. For simplicity, we consider the $L$-level un-decimal wavelet tight frame system without the down-sampling and up-sampling operators here. Let $h_{0}$ denote the mask associated with the scaling function and $\{h_{1}, \ldots , h_{n}\}$ denote the masks associated with other framelets. Denote

$$\begin{aligned} h^{(l)}_{j}=\underbrace{h_{0}*h_{0}*\cdots *h_{0}}_{l-1}*h_{j} \end{aligned}$$

(6)

where $*$ denotes the discrete convolution operator. Then $W_{l,j}$ corresponds to the Toeplitz-plus-Hankel matrix that represents the convolution operator $h^{(l)}_{j}$ under Neumann boundary condition. For the general definition of $W_{l,j}$ refer to [17–19]. Besides, We can also use $\alpha$ to denote the frame coefficients, i.e., $\alpha =W u$, where $\alpha _{l,j}=W_{l,j}u$.

In what follows, we further recall the recently proposed image restoration models based on framelets. Due to the redundancy of the wavelet tight frames ($W W^{T}\ne I$), there are several different frame-based models proposed in the literature, including the analysis model, the synthesis model, and the balance model. Detailed description of these methods can be found in [6, 7]. Typically, the balanced model for framelet-based image restoration can be described as

$$\begin{aligned} \min _{\alpha }\frac{1}{2}\Vert KW^{T}\alpha -f\Vert _{2}^{2}+ \frac{\kappa }{2}\Vert \left( I-W W^{T}\right) \alpha \Vert _{2}^{2}+\left\| \sum _{l=0}^{L-1}\left( \sum _{j\in {\fancyscript{I}}}\lambda _{l,j}|\alpha _{l,j}|^{p}\right) ^{1/p}\right\| _{1} \end{aligned}$$

(7)

where $p=1$ or 2, $0\le \kappa \le \infty$, and $\lambda _{l,j}>0$ is a scalar parameter. Note that the last two terms in (7) balance the smoothness of the image and the sparsity of the transformed coefficients. The synthesis and analysis models can be regarded as two extreme cases of (7). While $\kappa =0$, the balanced model (7) only penalizes the sparsity of framelet coefficients. This is called the synthesis model. While $\kappa =\infty$, (7) can be reformulated as

$$\begin{aligned} \min _{u}\frac{1}{2}\Vert K u-f\Vert _{2}^{2}+\left\| \sum _{l=0}^{L-1}\left( \sum _{j\in {\fancyscript{I}}}\lambda _{l,j}|W_{l,j}u|^{p}\right) ^{1/p}\right\| _{1}. \end{aligned}$$

(8)

This is called the analysis model. Though the formulas of the three models are different from each other, numerical experiments in [6] demonstrate that the quality of the recovered images by these models is approximately comparable.

All the frame based models mentioned above utilize the $l_1$-norm of frame coefficients as the regularizer. In order to recover images with more sharp edges, the $l_q$ quasi-norm with $0\le q<1$ was further investigated. Recently, the authors in [25] proposed to use the $l_{0}$ “norm” instead of the $l_{1}$ norm in the analysis model:

$$\begin{aligned} \min _{u}\frac{1}{2}\Vert K u-f\Vert _{2}^{2}+\sum _{\mathbf{i }} \lambda _{\mathbf{i }} \left\| (W u)_{\mathbf{i }}\right\| _{0} \end{aligned}$$

(9)

where multi-index $\mathbf{i }$ is used and $(W u)_{\mathbf{i }}$ denotes the value of $W u$ at a given pixel location within a certain level and band of wavelet frame transform. $\lambda _{\mathbf{i }}>0$ is the corresponding regularization parameter. The $l_{0}$ “norm” $\Vert w\Vert _{0}$ is defined to be the number of the non-zero elements of $w$. Its proximity operator can be easily computed by the hard-thresholding operator.

In literature [25], an algorithm called PD method was proposed to solve the $l_{0}$ minimization problem (9). Recently, a more efficient algorithm, called MDAL method [27], was further developed for solving the same problem. It can be seen as an extension of the doubly augmented Lagrangian (DAL) method [41, 42] to handle the non-convex regularizer such as $l_{0}$ minimization. The minimization problem (9) can be rewritten as a constrained form:

$$\begin{aligned} \min _{u}\frac{1}{2}\Vert K u-f\Vert _{2}^{2}+ \Vert \lambda \cdot \alpha \Vert _{0}, ~~~{\text {s.t.}}~~~\alpha =W u \end{aligned}$$

(10)

where $\Vert {\varvec{\lambda }}\cdot \alpha \Vert _{0}=\sum _{\mathbf{i }}\lambda _{\mathbf{i }} \left\| (\alpha )_{\mathbf{i }}\right\| _{0}$. The DAL method applied to (10) can be formulated as

$$\begin{aligned} \left\{ \begin{array}{lll} u^{k+1} = {\text {arg}}\min \limits _{u}\frac{1}{2}\Vert K u-f\Vert _{2}^{2} + \frac{\mu }{2}\Vert W u-\alpha ^{k}+b^{k}\Vert _{2}^{2} + \frac{\gamma }{2}\Vert u-u^{k}\Vert _{2}^{2}, &{}~ &{}~ ~\\ \alpha ^{k+1} = {\text {arg}}\min \limits _{\alpha }\Vert \lambda \cdot \alpha \Vert _{0} + \frac{\mu }{2}\Vert \alpha -\left( W u^{k+1}+b^{k}\right) \Vert _{2}^{2} + \frac{\gamma }{2}\Vert \alpha -\alpha ^{k}\Vert _{2}^{2}, &{}~ &{}~ ~\\ b^{k+1}=b^{k} + W u^{k+1}-\alpha ^{k+1}&{} ~ &{}~ \end{array} \right. \end{aligned}$$

(11)

where $\mu >0$ is a penalty parameter, and the parameter $\gamma >0$ controls the regularity of the iterative sequence.

Although applying the DAL algorithm (11) to solve the minimization problem (10) seems to be reasonable, numerical examples in literature [27] demonstrate that the iteration sequence generated by (11) may be unstable, or the convergence speed is very slow. Therefore, the authors there proposed to utilize the arithmetic means of the solution sequence, denoted by

$$\begin{aligned} \bar{u}^{k}=\frac{1}{k+1}\sum _{j=0}^{k}u^{k}, ~~\bar{\alpha }^{k}=\frac{1}{k+1}\sum _{j=0}^{k}\alpha ^{k}, \end{aligned}$$

as the output, instead of the sequence $(u^{k}, \alpha ^{k})$ itself. Due to the fact that the arithmetic means are treated as the actual output of the algorithm, it is called the mean doubly augmented Lagrangian (MDAL) method. In literature [43], the convergence of the arithmetic means of the sequence generated by DAL method has been investigated for the convex minimization problems. However, these results cannot be applied for the MDAL method here due to the non-convexity of the $l_{0}$ “norm”. Fortunately, numerical experiments show that the sequence $(\bar{u}^{k}, \bar{\alpha }^{k})$ generated by the MDAL method is really convergent, and both the convergence speed and the quality of recovery are superior to those of the PD method [27].

3 Proposed Model Based on $l_0-l_2$ Regularizer

In the above section, we briefly review the notion of framelet and the $l_{0}$-based optimization model for image restoration. Note that the frame coefficients can be obtained by the discrete convolution operator such as that defined in (6), which implies that they in fact represent the results of different types of finite-difference operators defined in the neighborhood. Therefore, the sparse prior of images under certain wavelet frame systems can only reflect the local features of image information. In fact, some of the textures and finer details contained in the small frame coefficients may be mistakenly removed by the $l_{0}$-minimization problem with respect to the frame coefficients (see the experiments below for details).

In order to estimate the frame coefficients that contain the textures and finer details more exactly, one strategy is to obtain a reasonable prior of the non-zero coefficients. Inspired by the recent works [33, 34] for patch-based sparse representation models, we propose the following nonlocal estimation of the frame coefficients. Consider the frame coefficients of the image $u$ in band $j$ at level $l$, which is denoted by $\alpha _{l, j}=W_{l, j} u$. By the definition of the operator $W_{l, j}$ shown in Sect. 2, we infer that the coefficients $\alpha _{l, j}$ is obtained by convoluting the image $u$ with the operator $h_{j}^{(l)}$. It is well known that the convolution is performed in the local image regions. Therefore, for any indexes $\mathbf{k }$ and $\mathbf{t }$, while the neighborhoods around the pixels $\mathbf{k }$ and $\mathbf{t }$ of the image are similar, we have that

$$\begin{aligned} \alpha _{l, j}(\mathbf{k })\approx \alpha _{l, j}(\mathbf{t }). \end{aligned}$$

(12)

The relation in (12) means that the similarity of two frame coefficients can be measured by the similarity of the corresponding image patches, and hence one frame coefficient corresponding to a given patch can be estimated by the frame coefficients obtained by the similar patches.

Denote the image patch centered at the pixel $\mathbf{k }$ as $\mathbf{u }_{\mathbf{k }}$, which is defined by $\mathbf{u }_{\mathbf{k }}=\{u_{\mathbf{q }}, ~\mathbf{q }\in {\fancyscript{N}}(\mathbf{k })\}$. Here ${\fancyscript{N}}(\mathbf{k })$ denotes a neighborhood of the pixel $\mathbf{k }$. Then from a collection of $m$ similar patches $\left\{ \mathbf{u }_{\mathbf{k }_{1}}, \ldots , \mathbf{u }_{\mathbf{k }_{m}}\right\}$, the nonlocal estimation of $\alpha _{l, j}(\mathbf{k })$ can be given by

$$\begin{aligned} \beta _{l, j}(\mathbf{k })= \sum _{i=1}^{m}w_{i}\alpha _{l, j}(\mathbf{k }_{i}) \end{aligned}$$

(13)

where $w_{i}$ is the weight based on patch similarity. Similarly to the nonlocal means filter [28], we set the weight to be inversely proportional to the distance between the patches $\mathbf{u }_{\mathbf{k }}$ and $\mathbf{u }_{\mathbf{k }_{i}}$, i.e.,

$$\begin{aligned} w_{i} = \frac{1}{C}{\text {exp}}\left( -\Vert \mathbf{u }_{\mathbf{k }_{i}} -\mathbf{u }_{\mathbf{k }}\Vert _{2}^{2}/h\right) \end{aligned}$$

where $h$ is a filtering parameter and $C$ is the normalizing factor.

Through combining the sparse prior and the nonlocal estimation of the frame coefficients, we propose the following wavelet frame based image restoration model

$$\begin{aligned} \min _{u, \beta }\frac{1}{2}\Vert K u-f\Vert _{2}^{2}+\sum _{\mathbf{i }} \lambda _{\mathbf{i }} \left\| (W u)_{\mathbf{i }}\right\| _{0} + \frac{\nu }{2} \Vert W u-\beta \Vert _{2}^{2} \end{aligned}$$

(14)

where $\beta =[\beta _{l, j}]_{0\le l\le L-1,j\in {\fancyscript{I}}}$ denotes the nonlocal estimation of $W u$, and its optimal value can be estimated by the formula (13). Here we use the $l_{2}$ norm to measure the approximation between $W u$ and its nonlocal estimation for the convenience of computation. The variables $u$ and $\beta$ are alternatively updated in the proposed algorithm.

Since the optimal value of $\beta$ is obtained by (13), we firstly fix the value of $\beta$, and hence the minimization problem (14) can be rewritten as a constrained problem as follows:

$$\begin{aligned} \min _{u, \alpha }\frac{1}{2}\Vert K u-f\Vert _{2}^{2}+\sum _{\mathbf{i }} \lambda _{\mathbf{i }} \left\| (\alpha )_{\mathbf{i }}\right\| _{0} + \frac{\nu }{2} \Vert \alpha -\beta \Vert _{2}^{2}, ~~~{\text {s.t.}}~~~\alpha =W u. \end{aligned}$$

(15)

Similarly to literature [27], we can use the MDAL method to solve the proposed $l_0-l_2$ minimization problem. The DAL method applied to (15) is reformulated as

$$\begin{aligned} \left\{ \begin{array}{lll} u^{k+1} = {\text {arg}}\min \limits _{u}\frac{1}{2}\Vert K u-f\Vert _{2}^{2} + \frac{\mu }{2}\Vert W u-\alpha ^{k}+b^{k}\Vert _{2}^{2} + \frac{\gamma _{1}}{2}\Vert u-u^{k}\Vert _{2}^{2}, &{}~ &{}~ ~\\ \alpha ^{k+1} = {\text {arg}}\min \limits _{\alpha }\Vert \lambda \cdot \alpha \Vert _{0} + \frac{\nu }{2} \Vert \alpha -\beta \Vert _{2}^{2} + \frac{\mu }{2}\Vert \alpha -\left( W u^{k+1}+b^{k}\right) \Vert _{2}^{2} + \frac{\gamma _{2}}{2}\Vert \alpha -\alpha ^{k}\Vert _{2}^{2}, &{}~ &{}~ ~\\ b^{k+1}=b^{k} + W u^{k+1}-\alpha ^{k+1}.&{} ~ &{}~ \end{array} \right. \end{aligned}$$

(16)

Each of the subproblems of (16) has a closed form solution. It is simple to show that (16) can be rewritten as

$$\begin{aligned} \left\{ \begin{array}{lll} u^{k+1} = \left( K^{T}K+\left( \mu +\gamma _{1}\right) I\right) ^{-1} \left( K^{T}f + \gamma _{1}u^{k} + \mu W^{T}\left( \alpha ^{k}-b^{k}\right) \right) , &{}~ &{}~ ~\\ \alpha ^{k+1} = {\fancyscript{H}}_{\lambda ,\nu ,\mu ,\gamma _{2}}\left( \beta , W u^{k+1}+b^{k}, \alpha ^{k}\right) , &{}~ &{}~ ~\\ b^{k+1}=b^{k} + W u^{k+1}-\alpha ^{k+1}&{} ~ &{}~ \end{array} \right. \end{aligned}$$

(17)

where the operator ${\fancyscript{H}}$ is a generalized hard-thresholding operator defined component-wisely as

$$\begin{aligned} \left( {\fancyscript{H}}_{\lambda ,\nu ,\mu ,\gamma }(x,y,z)\right) _{i}= {\left\{ \begin{array}{ll} 0, &{}\text{ if }\,\left| \frac{\nu x_{i}+\mu y_{i}+\gamma z_{i}}{\nu +\mu +\gamma }\right| <\sqrt{\frac{2\lambda _{i}}{\nu +\mu +\gamma }},\\ \frac{\nu x_{i}+\mu y_{i}+\gamma z_{i}}{\nu +\mu +\gamma }, &{}\text{ otherwise }. \end{array}\right. } \end{aligned}$$

In what follows, we further discuss how to compute the nonlocal estimation $\beta$ while the variables $u$ and $\alpha =W u$ are fixed. From the formula (13) we observe that the values of $\beta$ rely on the frame coefficients $\alpha$. In order to estimate the values of $\beta$, we use the the current estimate $\alpha ^{k}$ to approximate $\alpha$. Therefore, the values of $\beta$ at the $k+1$th iteration of the iterative method (17) can be updated by the following formula:

$$\begin{aligned} \beta _{l, j}(\mathbf{k })= \sum _{i=1}^{m}w_{i}\alpha ^{k}_{l, j}(\mathbf{k }_{i}). \end{aligned}$$

(18)

The nonlocal estimation of each frame coefficient will bring an extra computational burden for the iteration process. However, we only need the nonlocal prior of the coefficients that contain the textures or finer structures, and hence the computation of the nonlocal estimation of most frame coefficients may be unnecessary.

For example, consider the Barbara image which is convolved by a $5\times 5$ Gaussian blur kernel with standard deviation $\sigma _{b}=2$, and contaminated by Gaussian noise of standard deviation $\sigma =3$. We use the proposed model (14) to recover the blurred image. For the setting of parameters of the DAL method shown in (16), refer to the numerical experiments in Sect. 4. Figure 1 shows the frame coefficients $\alpha ^{k}_{l, j}$ with $l=1$, $j=(3, 2)$ and $k=10, 11, 12$. In this figure, the values of frame coefficients are presented in gray scale where brightness is measured in a range of [0, 1]. It is observed that the difference of the support sets of the frame coefficients between two successive iteration values is very small. Specifically, we can define the diversity function

$$\begin{aligned} \chi _{l,j}^{k}=\frac{|{\text {supp}}\left( \alpha ^{k}_{l, j}\right) \cup {\text {supp}}\left( \alpha ^{k+1}_{l, j}\right) |-|{\text {supp}}\left( \alpha ^{k}_{l, j}\right) \cap {\text {supp}}\left( \alpha ^{k+1}_{l, j}\right) |}{m\times n} \end{aligned}$$

to quantitatively measure the difference between two support sets. Here ${\text {supp}}(\cdot )$ denotes the support set, $|\cdot |$ denotes the number of elements in the set, and $m\times n$ represents the size of the image. Through direct computation we obtain that $\chi _{l,j}^{10}=2.6\,\%$ (reflect the difference of the support sets shown in Fig. 1a, b) and $\chi _{l,j}^{11}=2.57\,\%$ (reflect the difference of the support sets shown in Fig. 1b, c). These results demonstrate that the change between two successive support sets is really small. Therefore, we can use the current support set of $\alpha ^{k}$ to define an active set which provides a permissible range for the updating of the nonlocal estimation $\beta$.

Fig. 1

The frame coefficients of band (3, 2) at level 1. a iter. num. = 10; b iter. num. = 11; c (a) iter. num. $=$ 12

Motivated by the above analysis, we introduce a new strategy to further accelerate the proposed algorithm. Denote the active set

$$\begin{aligned} {\fancyscript{I}}^{k}=\left\{ i:~~\left| \frac{\nu \beta ^{e}_{i}+\mu \left( W u^{k}+b^{k-1}\right) _{i}+\gamma \alpha ^{k-1}_{i}}{\nu +\mu +\gamma }\right| >c\cdot \sqrt{\frac{2\lambda _{i}}{\nu +\mu +\gamma }}\right\} \end{aligned}$$

(19)

where $\beta ^{e}$ denote the current estimation value of $\beta$ before the $k+1$th iteration, and $c\in (0, 1)$ is a scaling parameter used to restrict the range of the update of $\beta$ in a small set. Then we only need to update the values of $\beta$ in the active set ${\fancyscript{I}}^{k}$. Since this strategy limits the computation of nonlocal estimation values in a small set of the framelet domain, we call it the narrow-band technique in this paper. The numerical experiments in Sect. 4 will demonstrate that the introduction of the narrow-band technique obviously accelerates the proposed algorithm. The whole process of the proposed algorithm is summarized in Algorithm 1.

In the proposed algorithm, the parameter $T$ represents the update interval of the nonlocal estimation value $\beta$ in the iteration process. For the initialization of the nonlocal estimation $\beta$, we use the folowing the strategy. First, similarly to the literature [30, 47], we use the standard Tikhonov regularization method to obtain an initial image $\hat{u}$, and the similarity weight $\mathbf{w }$ can be computed according to $\hat{u}$. Then the initial $\beta ^{0}$ is obtained by

$$\begin{aligned} \beta ^{0}_{l, j}\left( \mathbf{k }\right) = \sum _{i=1}^{m}w_{i}\left( W_{l, j}\hat{u}\right) \left( \mathbf{k }_{i}\right) . \end{aligned}$$

Though the MDAL method is superior to the previous PD method, the convergence to a local minimizer of the sequence generated from the MDAL method for the $l_{0}$ minimization problem is still an open problem. Fortunately, the numerical experiments in literature [27] demonstrate that the algorithm is really convergent towards a stable solution. Though a extra nonlocal prior is added in the model via the form of $l_{2}$ norm, the convergence of our proposed algorithm can still be guaranteed. In fact, it is observed that the iterative algorithm for the $l_{0}-l_{2}$ minimization problem can achieve the same accuracy with much less iteration number compared to that for the $l_{0}$ minimization problem. For more details refer to the experiments in the next section.

4 Numerical Experiments

In this section, we compare the proposed algorithm (shown as Algorithm 1) with the PBOS algorithm for the non-local TV model [30], the MDAL method for the $l_{0}$ minimization problem [27] and the Iterative Decoupled Deblurring BM3D (IDD-BM3D) algorithm [46]. The specific image restoration task that we consider here is image deblurring. Both the quality of the recovery images and the computational costs of these algorithms are compared.

The codes of the proposed algorithm and the MDAL method for the $l_{0}$ minimization are written entirely in Matlab. The code of the PBOS algorithm for the nonlocal TV model is available at http://math.sjtu.edu.cn/faculty/xqzhang/html/code.html. Due to the fact that a large scale linear equations need to be solved in each inner iteration of the PBOS algorithm, the total computational costs are very high. Therefore, the main code of the PBOS algorithm is written in C with MATLAB calling interfaces. The code of the IDD-BM3D algorithm is available at http://www.cs.tut.fi/~foi/GCF-BM3D/BM3D.zip. The computationally most intensive parts have been written in C++ due to the high computational complex. All the numerical examples are implemented under Windows XP and MATLAB 2009 running on a laptop with an Intel Core i5 CPU (2.8 GHz) and 8 GB Memory. In the following experiments, six standard natural images (see Fig. 2), which consist of complex components in different scales and with different patterns, are used for our test.

Fig. 2

Original images. a Barbara ($256\times 256$), b Lena ($256\times 256$), c house ($256\times 256$), d Baboon ($256\times 256$), e Goldhill ($256\times 256$), f Zebra ($250\times 167$)

4.1 The Setting of Parameters

In this section we discuss the selection of parameters of Algorithm 1 in details. The parameters $\mu , \gamma _{1}, \gamma _{2}$ are the penalty parameters of the MDAL algorithm. These parameters mainly affect the speed of convergence of their corresponding algorithm. Empirically, these parameters are not very sensitive to different types of images, blurs and noise levels. Optimal adjustments of these parameters may improve the results which are presented here. However, it will also reduce the practicality of the algorithm due to the estimation of the optimal parameters. Therefore, based on the suggestion in literature [27], we fix the penalty parameters $\mu =0.01$ and $\gamma _{1}=\gamma _{2}=0.003$ for Algorithm 1.

The setting of the regularization parameter $\lambda$ is the same as that in literature [27]. For the regularization parameter $\nu$ which controls the goodness of fit of the nonlocal estimation $\beta$, we adopt two selection strategy here. One is to use the fixed parameter, which is adjusted to be a suitable value such as $\nu =0.02$ used in the following experiments, in the sense that after many trials, this value gives the best restoration; the other is to use an adaptive parameter which is given by

$$\begin{aligned} \nu _{l,j}(\mathbf{k })=\frac{c_{0}}{\sqrt{\max \left( \sum _{i=1}^{m}|W_{l,j}u(\mathbf{k }_{i})-\beta _{l,j}^{e}(\mathbf{k })|_{2}^{2}, \sigma ^2\right) }} \end{aligned}$$

(20)

where $\beta _{l,j}^{e}$ denotes the current estimation of $\beta _{l,j}$ in the iteration process, and $c_{0}$ is a constant. Through many trials we choose $c_{0}=0.003$ in our experiments. Note that the denominator is a non-local approximation of the standard deviation of the coefficient $W_{l,j}u(\mathbf{k })$. For the texture-like image patch which has many repetition patterns, the non-local standard deviation of the coefficient corresponding to this patch will be small, and hence the value of $\nu$ will be large there, which means that the second regularization term $\Vert W u-\beta \Vert _{2}^{2}$ of model (14) plays the main role. Therefore, the values of $\nu$ defined by (20) changes according to the image contents. However, the computation of the adaptive regularization parameter also introduces extra computational cost and will reduce the efficiency of the proposed algorithm. A compromise choice is to compute the value of the adaptive parameter $\nu$ according to (20) just several times, and use this fixed value in the remaining iteration process. In our experiments, we observe that the recovery results of the compromise method which computes (20) for $\nu$ just once are even worse than those obtained by the algorithm with the fixed parameter, which maybe due to the reason that the estimation of the standard deviation of the coefficient $W_{l,j}u(\mathbf{k })$ shown in (20) is inexact in the first several iteration.

In what follows, we compare the performance of proposed algorithms with both selection strategies of the regularization parameter $\nu$, and the compromise method. For the computation of the nonlocal weights, we adopt the similar setting which is widely used in many previous literatures [33, 34, 36] related to this class of computation, i.e., the sizes of image patches and the searching window are fixed to be $5\times 5$ and $11\times 11$ respectively. The 15 nearest patches are used for the computation of the nonlocal prior, i.e., we set $m=15$. In fact, the performance of the proposed algorithm is unsensitive to the setting of these parameters. Similarly to previous literature such as [5, 7, 8, 27], linear B-spline framelet is adopted for our experiments. In order to reduce the computational cost, we fix the level of framelet decomposition to be 1 here. Two standard natural images called “Barbara” and “Zebra” are used for our comparison. We consider two blur kernel cases: one is a Gaussian blur [generated by the Matlab function “fspecial (‘gaussian’, 5, 2)”], the other is a 20 degree motion blur [generated by the Matlab function “fspecial (‘motion’, 10, 20)”]. The blurred images are contaminated by noises with the standard deviation $\sigma =3$ and 4. The performance of these algorithms is quantitatively measured by means of the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index [48]. The PSNR is expressed as

$$\begin{aligned} {\text {PSNR}}(u,u^{*})=-20\lg \left\{ \frac{\Vert u-u^{*}\Vert _2}{255 N}\right\} \end{aligned}$$

(21)

where $u$ and $u^{*}$ denote the original and restored images, respectively, and $N$ is the total number of pixels in the image $u$. Besides, we choose the following stopping criterion for the MDAL method:

$$\begin{aligned} \frac{\Vert \bar{u}^{k+1}-\bar{u}^{k}\Vert _{2}}{\Vert f\Vert _{2}}<5\times 10^{-4}. \end{aligned}$$

(22)

Table 1 shows the PSNR, SSIM values and iteration time of the proposed algorithms with different selection strategies of $\nu$. In the compromise choice strategy, the value of the parameter $\nu$ is computed according to (20) just nine times, and it is observed that the recovery results are more or less the same as those obtained by the algorithm with the adaptive parameter. Besides, we also observe from the results that though the performance of the algorithm is improved by the adaptive parameter $\nu$ defined by (20), the corresponding iteration time becomes longer than that with the fixed parameter due to the extra computation task of $\nu$.

Table 1 The comparison of the performance of the proposed algorithms with different selection strategy of $\nu$: the given values are PSNR (dB)/SSIM/CPU time (s)

In what follows, we further discuss the selection of the update parameter $T$ and the scaling parameter $c$. We use three standard natural images called “Barbara”, “lena” and “House” for our test. Two blur cases used in the above experiments are also tested here. First of all, we consider the impact of the update parameter $T$ on the proposed algorithm (we consider the case of the fixed parameter $\nu$, and the conclusion is also suitable for the adaptive parameter cases). Here we fix $c=0.6$ in Algorithm 1. Figure 3 shows the curves about the PSNR values vs. the update parameter $T$. The results of the Barbara image with different blur kernels are shown in Fig. 3a, and those of the House image are shown in Fig. 3b. It is observed that the quality of recovery images is degraded very quickly while $T>2$. Therefore, we choose $T=2$ in the following experiments, which means the value of the nonlocal estimation $\beta$ is recalculated every second iteration.

Fig. 3

The curves of the PSNR values (dB) versus $T$. a Barbara image convoluted by two blur kernels and added with noise of $\sigma =3$; b house image convoluted by two blur kernels and added with noise of $\sigma =4$

Second, we investigate the influence of the scaling parameter $c$ of Algorithm 1 on both the quality of the recovery images and the computational costs. Two images with different blur kernels and noisy levels are considered here. We choose $T=2$ in Algorithm 1. Table 2 lists the PSNR values and CPU time of the iterative algorithm with $c$ varying from $0.0$ to $1.0$. From the results we observe that it takes shorter computational time for bigger $c$ due to the fact that the corresponding active set ${\fancyscript{I}}^{k}$ become smaller. However, the corresponding PSNR values decrease as the values of $c$ increase. Considering both the quality of the recovery images and the computational time, we choose $c=0.6$ in the following experiments.

Table 2 The PSNR (dB) and CPU time (s) with different scaling parameters $c$

4.2 The Analysis of the Role of the $l_{2}$ Regularizer of the Proposed Model

Note that compared with the MDAL method for the $l_{0}$ minimization [27], a nonlocal prior of the frame coefficients in the form of the $l_{2}$-norm regularizer has been introduced in the proposed model. In this section we further discuss the role of the added $l_{2}$ regularizer in both the computational efficiency and recovery accuracy. For simplification we consider the algorithm with the fixed $\nu$, and the similar conclusion can be also obtained by the algorithm with the adaptive parameter cases.

Firstly, we compare the computational efficiency of the MDAL methods for the $l_{0}$ minimization [27] and the $l_{0}-l_{2}$ minimization problems (Algorithm 1). Figure 4 shows the evolution curves of the log error $\ln \frac{\Vert \bar{u}^{k+1}-\bar{u}^{k}\Vert _{2}}{\Vert f\Vert _{2}}$ for both algorithms. Here the error is shown in log scale for better visualization. From the plots we observe that Algorithm 1 converges faster than the MDAL method for the $l_{0}$ minimization problem (especially for the Barbara image which is rich in textures). This phenomenon can be explained as follows. Assume that part of the frame coefficients, such as those corresponding to the textures or finer details, are mainly recovered by the nonlocal estimator $\beta$, which is updated by

$$\begin{aligned} \beta =S \alpha ^{k} \end{aligned}$$

where $S$ is the similarly matrix composed by the normalized weights. It is obvious that $S$ is a stochastic matrices with nonnegative entries. By Perron-Frobenius theory of nonnegative matrices [44], $e_{1}=1$ is the unique eigenvalue of $S$ with maximum modulus, and the convergence rate of $S^{k}$ is determined by the subdominant eigenvalue $e_{2}<1$. Therefore, if the coefficients $\alpha$ on the index set ${\fancyscript{J}}$ is determined by the nonlocal estimation, i.e., $(\alpha ^{2k})_{{\fancyscript{J}}}\approx (S^{k}\alpha ^{0})_{{\fancyscript{J}}}$, then $\alpha ^{k}$ converges very quickly on the set ${\fancyscript{J}}$ due to the fast convergence rate of the operator $S^{k}$. This may lead to the acceleration of the whole iterative algorithm. It is worth noticing that the acceleration is more obvious for the Barbara image. The reason is that more textures, corresponding to the larger index set ${\fancyscript{J}}$ where $(\alpha ^{k})_{{\fancyscript{J}}}$ converges very fast, need to be recovered by the nonlocal estimation.

Fig. 4

The evolution curves of the error $\ln \frac{\Vert \bar{u}^{k+1}-\bar{u}^{k}\Vert _{2}}{\Vert f\Vert _{2}}$ for images with different blur kernels and noise levels. a, b Barbara image convoluted by Gaussian kernel and added with noises of $\sigma =3$ and $\sigma =4$ respectively; c, d Lena image convoluted by motion kernel and added with noises of $\sigma =3$ and $\sigma =4$ respectively

In the next, the recovery accuracy of the MDAL method for the $l_{0}$ minimization [27] and Algorithm 1 is compared. Figure 5 presents efficiency curves, i.e. plotting relative log errors to ground-truth (denoted by $\ln \frac{\Vert \bar{u}^{k}-u^{*}\Vert _{2}}{\Vert u^{*}\Vert _{2}}$) versus computational time for both algorithms. From the plots we observe that the results obtained by Algorithm 1 are closer to the original image. This is due to the more accurate recovery of frame coefficients that correspond to textures or details through the nonlocal estimation $\beta$ of coefficient values.

Fig. 5

The evolution curves of the error $\ln \frac{\Vert \bar{u}^{k}-u^{*}\Vert _{2}}{\Vert u^{*}\Vert _{2}}$ for images with different blur kernels and noise levels. a, b Zebra image convoluted by Gaussian kernel and added with noises of $\sigma =3$ and $\sigma =4$ respectively; c, d Barbara image convoluted by motion kernel and added with noises of $\sigma =3$ and $\sigma =4$ respectively

In order to see this point more clearly, we also show the values of frame coefficients in Fig. 6. Figure 6a, d present certain sub-bands of the original Zebra and Barbara images. The light gray regions refer to large values of coefficients, whereas dark gray represents zones where coefficients is small. The Zebra image is blurred by Gaussian kernel and added with noises of $\sigma =4$, and the Barbara image is blurred by motion kernel and added with noises of $\sigma =3$. The corresponding sub-bands of the recovered images by the $l_0$ minimization are shown in Fig. 6b, e; and those by Algorithm 1 are presented in Fig. 6c, f. It is observed that though the sub-bands generated by the $l_0$ minimization is sparser than those produced by Algorithm 1, they are farther away from the ground-truth. This is due to the fact that a transformed image of a framelet transform is not truly sparse, but approximated sparse.

Fig. 6

a The frame coefficients of band $(1,3)$ at level 1 of the original Barbara image, b the corresponding coefficients of the recovered image by the $l_{0}$ minimization [27], c the corresponding coefficients of the recovered image by Algorithm 1, d the frame coefficients of band $(3,3)$ at level 1 of the original Zebra image, e the corresponding coefficients of the recovered image by the $l_{0}$ minimization [27], f the corresponding coefficients of the recovered image by Algorithm 1

4.3 Comparison with Other Methods

In this subsection, we report the experimental results, comparing the proposed algorithm with the MDAL method for the $l_{0}$ minimization problem, the PBOS algorithm for the nonlocal TV model and the IDD-BM3D algorithm.

First of all, we test the performance of the three algorithms including the $l_{0}$ minimization algorithm, the PBOS algorithm and Algorithm 1 with fixed $\nu$. Table 3 lists the PSNR, SSIM values and CPU time of the iterative process of different algorithms for images with different blur kernels and noise levels. Note that the PBOS algorithm for the nonlocal TV model is running in C with MATLAB calling interfaces due to the large computational burden. Besides, we also find that the PBOS algorithm cannot satisfy the stopping criterion based on the error between two successive iteration values such as $\Vert u^{k+1}-u^{k}\Vert _{2}/\Vert f\Vert _{2}<5\times 10^{-4}$ for many iterations, and hence the stopping criterion of $\Vert K u^{k}-f\Vert _{2}<\sigma$ which were adopted in literature [30] is also demanded here, i.e., the iterative process stops while one of the two criterions is satisfied. In this experiments we fix the level of framelet decomposition to be $L=2$ for the MDAL for $l_{0}$ minimization [27], and consider two cases of $L=1$ and 2 for Algorithm 1. We observe that the proposed algorithm has an overall better performance than the other two methods in terms of implementation time, SSIM and PSNR values. The iteration time of Algorithm 1 with $L=1$ is even less than that of the PBOS algorithm, which is implemented in C code. On the other hand, it is also noted that Algorithm 1 with $L=2$ can improve the quality of recovery images in most cases, but the corresponding computational time is also longer.

Table 3 The comparison of the performance of different algorithms under mild blur condition: the given values are PSNR (dB)/CPU time(s)

In what follows, we further analyze the advantage of the proposed algorithm through the image “Barbara”. The recovered results of the Barbara image are shown in Fig. 7. Figure 7a is the original Barbara image. The image is convoluted by 20-degree motion kernel and contaminated by noise with $\sigma =3.0$ is presented in Fig. 7b. Figure 7c–f show the recovered images by these algorithms. It is observed that the proposed algorithm has the potential to recover the piecewise smooth regions and the textures simultaneously. In order to make the comparison more clear, we zoom into two classical regions of the Barbara image, which represent the smooth regions and textures, in Figs. 8 and 9 respectively.

Fig. 7

a The original Barbara image, b the blurry and noisy image: Motion blur with noise of $\sigma =3.0$, c the image restored by nonlocal TV model, d the image restored by MDAL for $l_{0}$ minimization problem, e the image restored by Algorithm 1 with $L=1$, f the image restored by Algorithm 1 with $L=2$ (Color figure online)

Fig. 8

a The detail in the ramp region (red) corresponding to Fig. 7c, b the detail in the ramp region (red) corresponding to Fig. 7d, c the detail in the ramp region (red) corresponding to Fig. 7e, d the detail in the ramp region (red) corresponding to Fig. 7f

Fig. 9

a The detail in the ramp region (blue) corresponding to Fig. 7c, b the detail in the ramp region (blue) corresponding to Fig. 7d, c the detail in the ramp region (blue) corresponding to Fig. 7e, d the detail in the ramp region (blue) corresponding to Fig. 7f

From the results in Fig. 8, we observe that the texture details on the scarf are sharper in Fig. 8a, c, d. The reason is that nonlocal priors are introduced in the nonlocal TV model, and the proposed method through the $l_{2}$ norm. It is well known that the nonlocal methods have the ability of recovering the textures. However, the wavelet frame coefficients have the better capability of sparsely approximating piecewise smooth regions than the nonlocal filter. Therefore, it is noticed from Fig. 9 that the smooth regions can be better recovered by the frame-based image restoration methods than the nonlocal TV model. Specifically, we find that many artifacts or outliers appear in Fig. 9a, rather than in Fig. 9b–d.

Figure 10 presents the recovered results of the House image that is degraded by 20-degree motion kernel and Gaussian noise of $\sigma =4.0$. It is observed that many false contouring or artifacts appear in Fig. 10c, and not in Fig. 10d–f. For a better visualization, we also zoom parts of the results in Fig. 11. it can be seen clearly from the results that some artifacts also appear in Fig. 11b due to the over-sparsity of the frame coefficients based on the only $l_0$ minimization. This shortcoming is overcome by our proposed algorithm.

Fig. 10

a The original House image, b the blurry and noisy image: Motion blur with noise of $\sigma =4.0$, c the image restored by nonlocal TV model, d the image restored by MDAL for $l_{0}$ minimization problem, e the image restored by Algorithm 1 with $L=1$, f the image restored by Algorithm 1 with $L=2$ (Color figure online)

Fig. 11

a The detail in the ramp region (red) corresponding to Fig. 10c, b the detail in the ramp region (red) corresponding to Fig. 10d, c the detail in the ramp region (red) corresponding to Fig. 10e, d the detail in the ramp region (red) corresponding to Fig. 10f

The results of the Zebra image are shown in Fig. 12. The image is degraded by $11\times 11$ Gaussian kernel and contaminated by noise of $\sigma =3.0$. To illustrate the advantage of the proposed algorithm, we compare one smooth-like and one texture-like regions of the recovery images obtained by different algorithms. The corresponding zooming versions are presented in Fig. 13. For the smooth-like regions, it is noted that several artifacts (bright spots) appear in Fig. 13b, but not in Fig. 13c–e; for the texture-like regions, we observe that the stripes on the neck of the zebra can be recovered better by our method than by the $l_0$ minimization algorithm.

Fig. 12

a The original Zebra image, b the blurry and noisy image: $11\times 11$ Gaussian blur with noise of $\sigma =3.0$, c the image restored by nonlocal TV model, d the image restored by MDAL for $l_{0}$ minimization problem, e the image restored by Algorithm 1 with $L=1$, f the image restored by Algorithm 1 with $L=2$ (Color figure online)

Fig. 13

a The detail of original image (red), b the detail in the ramp region (red) corresponding to Fig. 12c, c that corresponding to Fig. 12d, d that corresponding to Fig. 12e, e that corresponding to Fig. 12f, f the detail of original image (red), g the detail in the ramp region (blue) corresponding to Fig. 12c, h that corresponding to Fig. 12d, i that corresponding to Fig. 12e, j that corresponding to Fig. 12f

Finally, we compare the proposed algorithm with the nonlocal patch-wise image deblurring method–IDD-BM3D algorithm [46], which is one of the state of the art in this field. Due to the high computational complex of the patch-wise methods, the code of IDD-BM3D algorithm is written in C++ and called as a preparsed version in the form of p-file. To guarantee the performance of IDD-BM3D algorithm, an two-stage approach of BM3DDEB [45] is used as an initial estimation. Two blur kernel cases are considered here. One is a Gaussian blur generated by fspecial(gaussian,9,2), the other is a 30 degree motion blur generated by fspecial(motion,15,30). Table 4 lists the PSNR, SSIM values and CPU time of the iterative process of different algorithms for several types of images, blurs and noise levels. In this table, “Algorithm 1, fixed” and “Algorithm 1, adaptive” represent Algorithm 1 with fixed and adaptive $\nu$ respectively. From the results we observe that the recovery quality of IDD-BM3D algorithm overall outperform the proposed algorithm. However, it requires much more computational time due to the high computational complex.

Table 4 The comparison of the performance of different algorithms under mild blur condition: the given values are PSNR (dB)/CPU time(second)

5 Conclusion

In this paper, we propose a new wavelet frame based image restoration model (14) which combines the the sparsity and nonlocal prior of frame coefficients. In the proposed model, the penalization of the $l_{0}$ “norm” of frame coefficients plays the role of recovering both the smoothness of homogeneous regions and the sharpness of edges, and the penalization of the $l_{2}$ norm related to the nonlocal estimator is used to preserve the coefficients that contain textures and finer details. The MDAL method is adopted to solve the $l_{0}-l_{2}$ minimization problem. Besides, a narrow-band technique is introduced to further improve the computational efficiency of the proposed algorithm. Numerical examples demonstrate that the proposed algorithm outperforms the recently proposed MDAL method for $l_{0}$ minimization problem in terms of both efficiency and the quality of the restored images. In addition, it is also superior to the nonlocal TV model especially in the field of preserving the homogeneous regions.