1 Introduction

Super-resolution (SR) image reconstruction refers to a process that produces a high-resolution (HR) image from a sequence of observed low-resolution (LR) images that are noisy, blurred, and downsampled [1, 2]. Image SR has a variety of applications, including remote sensing [3, 4], video frame freezing, medical diagnostics [5], and military information gathering, etc. The SR problem was first proposed by Tsai and Huang [6] in the frequency domain. They proposed a formulation for the reconstruction of a HR image from a set of undersampled, aliased but noise-free LR images. Their formulation was extended by Kim et al. [7] to consider observation noise as well as the effects of spatial blurring. Then, Kim and Su [8] extended their work by considering different blurs for each LR image. In the spatial domain, typical reconstruction methods include interpolation [9, 10], iterative back projection (IBP) [11], projection onto convex sets (POCS) [12–14], Bayesian/maximum a posteriori (MAP) [15–18], adaptive filtering [19], and sparse coding [20].

Before SR image reconstruction can be carried out, blur identification must be performed to estimate the blurring functions in the image formation process. The precise estimation of the blurring function is very important for the reconstruction of the HR image [21]. Most of the SR image reconstruction algorithms assume a priori known blur [15–18]. However, in many real applications, this assumption is impractical as it is difficult to predict the blur precisely. Consequently, this motivates the study of blind SR image reconstruction, which is a process to perform HR image reconstruction with limited or no knowledge of the blurring function [22]. A few investigations have been carried out to estimate the HR image and blurring function simultaneously, to reduce the effect of blur estimation error. Authors in [23–25] proposed blind SR that can handle parametric blur models with one parameter. This restriction is, unfortunately, very limiting for most real applications. To our knowledge, early attempts at theoretical blind SR with an arbitrary blurring function have appeared in [26, 27]. Then, He et al. [22] developed a new soft blur learning scheme, which attempts to integrate the parametric information of the blurring function into the algorithm. Sroubek et al. [28] proposed a unifying method that simultaneously estimates the volatile blurs and HR image without any prior knowledge of the blurs and the original image.

Image registration is another process that must be performed to estimate the subpixel shifts between LR images before image SR reconstruction, and plays a critical role in SR image reconstruction [29, 30]. Due to the presence of aliasing in the captured LR images, most existing registration algorithms for aliased images, such as the algorithms described in [31–33], still experience subpixel errors [34]. In the case of non-blind SR where blurring functions are assumed known, many works have been carried out with the emphasis on reducing the effect of registration errors. The simultaneous image registration and reconstruction approach is a quite important approach. Tom and Katsaggelos [35] developed a simultaneous registration and reconstruction approach, in which they formulated the SR problem in the form of a maximum likelihood problem and solved it using the expectation–maximization algorithm. Hardie et al. [36] developed an approach within the MAP framework to simultaneously estimate the image registration parameters and the HR image. Segall et al. [37, 38] presented a noteworthy approach involving the joint estimation of dense motion vectors and HR images for compressed video. Woods et al. [39] presented complex stochastic methods in which the parameters of registration, noise and image statistics are estimated jointly based on the available observations. In [40], Chung et al. proposed a nonlinear cost function, and estimated the registration parameters and the HR image using the Gauss–Newton method. Considering a more generic motion model that includes both translation and rotation, He et al. [34] proposed an iterative scheme based on a non-linear least squares method to estimate the motion parameters and the HR image simultaneously. More recently, Tian and Yap presented a new framework for joint image registration and HR image reconstruction from multiple LR observations with zooming motion.

While in the case of blind SR, there are few reconstruction methods considering registration errors. The conventional blind SR algorithms are performed in two disjoint stages, namely, (1) image registration from LR images, followed by (2) simultaneous estimation of both the HR image and blurring function. Generally speaking, these blind SR algorithms ignore registration errors and assume that the estimated motion parameters of existing registration methods are error free. Obviously, this assumption is impractical in many real applications. Thus, the registration errors will affect the subsequent blind SR image reconstruction to certain extent.

In view of this, this paper proposes a new blind SR reconstruction method considering image registration errors, which establishes a MAP framework for joint image registration, blur identification, and HR image reconstruction. As image registration, blur identification, and SR reconstruction are mutually interdependent and influence each other, an ideal approach is to address them simultaneously. To the best of our knowledge, to date, only two among these three processes have been simultaneously addressed, such as the joint estimation of image registration and reconstruction [30, 34–40] and that of arbitrary blur and image reconstruction [22, 26–28]. The main contribution of this paper is that we integrate image registration into blind SR image reconstruction. Thus, the registration information is iteratively updated along with the progressively estimated blurring function, and HR image, in a cyclic manner. This is more promising as more accurate registration parameters can be determined, thereby improving the performance of the blur identification and SR reconstruction. This algorithm reinforces the interdependence of the registration parameters, blurring function, and HR image in a mutually beneficial manner. Experimental results show that the proposed joint method is effective in performing image registration, blur identification, and SR reconstruction for simulated, as well as real-life images.

The remainder of the paper is organized as follows. In Sect. 2, the SR observation model is described. The joint MAP estimation problem is formulated in Sect. 3. In Sect. 4, the joint optimization procedure to solve the registration parameters, blurring function, and HR image is presented. Experimental results are provided in Sect. 5, and Sect. 6 concludes the paper.

2 Observation Model

The image observation model is employed to relate the desired referenced HR image to the observed LR images. Typically, the imaging process involves warping, followed by blurring and downsampling to generate LR images from the HR image. Let the underlying HR image be denoted in the vector form by \( {\mathbf{z}} = \left[ {z_{1} ,z_{2} ,{ \ldots },z_{{L_{1} N_{1} \times L_{2} N_{2} }} } \right]^{T} \), where L 1 N 1 × L 2 N 2 is the HR image size. Letting L 1 and L 2 denote the downsampling factors in the horizontal and vertical directions, respectively, each observed LR image has the size N 1 × N 2. Thus, the LR image can be represented as \( {\mathbf{g}}_{k} = \left[ {g_{k,1} ,g_{k,2} ,{ \ldots },g_{{k,N_{1} \times N{}_{2}}} } \right]^{T} \), where \( k = 1,2,{ \ldots },P \), with P being the number of LR images. Assuming that each observed image is contaminated by additive noise, the observation model can be represented as [1]

$$ {\mathbf{g}}_{k} = {\mathbf{DB}}_{k} {\mathbf{M}}_{k} (s){\mathbf{z}} + {\mathbf{n}}_{k}, $$
(1)

where M k (s) is the warp matrix with the size of L 1 N 1 L 2 N 2 × L 1 N 1 L 2 N 2, s represents the registration information, B k represents the camera blur matrix also of size L 1 N 1 L 2 N 2 × L 1 N 1 L 2 N 2, D is a N 1 N 2 × L 1 N 1 L 2 N 2 downsampling matrix, and n k represents the N 1 N 2 × 1 noise vector. Figure 1 illustrates (1).

Fig. 1
figure 1

Block diagram illustration of the observation model (1), where the desired HR image is at the extreme left, and the observed image at the extreme right [18]

Full size image

Similar to most work on SR [1, 22], it is assumed in this paper that the imaging blur is spatial shift-invariant and the warping between different LR images is translational. Using the commutative property of convolution [41], (1) may be written as

$$ {\mathbf{g}}_{k} = {\mathbf{DM}}_{k} (s){\mathbf{B}}_{k} {\mathbf{z}} + {\mathbf{n}}_{k} = {\mathbf{DM}}_{k} (s){\mathbf{Zb}}_{k} + {\mathbf{n}}_{k}, $$
(2)

where B k z, the blurry version of the HR image, can also be written as Zb k , by introducing a L 1 N 1 L 2 N 2 × r 1 r 2 matrix Z, which is constructed from image z. b k is the blur kernel with the size of r 1 r 2 × 1; here r 1 × r 2 represents the support of the blurring function. Let the full set of P LR images, blurring functions and registration parameters be denoted by \( {\mathbf{g}} = \left\{ {{\mathbf{g}}_{1} ,{\mathbf{g}}_{2} ,{ \ldots },{\mathbf{g}}_{P} } \right\} \), \( {\mathbf{b}} = \left\{ {{\mathbf{b}}_{1} ,{\mathbf{b}}_{2} ,{ \ldots },{\mathbf{b}}_{P} } \right\}, \) and \( {\mathbf{s}} = \left\{ {{\mathbf{s}}_{1} ,{\mathbf{s}}_{2} ,{ \ldots },{\mathbf{s}}_{P} } \right\} \), respectively. By stacking (2), we get

$$ {\mathbf{g}} = {\mathbf{DM}}(s){\mathbf{Bz}} + {\mathbf{n}} = {\mathbf{DM}}(s){\mathbf{Zb}} + {\mathbf{n}}. $$
(3)

3 Problem Formulation

The purpose is to realize the joint MAP estimate of HR image z, blurring functions b, and registration information s, given the observed LR images g. The estimate can be computed by

$$ {\hat{\mathbf{z}}},{\hat{\mathbf{b}}},{\hat{\mathbf{s}}} = \arg \hbox{max} \left\{ {p({\mathbf{z}},{\mathbf{s}},{\mathbf{b}}|{\mathbf{g}})} \right\}. $$
(4)

Applying Bayes’ rule, after some manipulation, we can get

$$ {\hat{\mathbf{z}}},{\hat{\mathbf{b}}},{\hat{\mathbf{s}}} = \arg \hbox{max} \left\{ {\log p({\mathbf{g}}|{\mathbf{z}},{\mathbf{b}},{\mathbf{s}}) + \log p({\mathbf{z}}) + \log p({\mathbf{b}}) + \log p({\mathbf{s}})} \right\}. $$
(5)

The central task then becomes constructing the four probability density functions (PDFs) so as to enable the MAP estimation. The PDF p(g|z, b, s) provides a measure of the conformance of the estimated image to the observed images, according to the image observation model. It is determined by the probability density of the noise vector in (3). Assuming that the noise n is additive white Gaussian noise (AWGN) with variance σ 2, the likelihood p(g|z, b, s) can be given by

$$ p({\mathbf{g}}|{\mathbf{z}},{\mathbf{b}},{\mathbf{s}}) = \frac{1}{{C_{1} }}\exp \left( { - \frac{{\left\| {{\mathbf{g}} - {\mathbf{DBM}}(s){\mathbf{z}}} \right\|^{2} }}{{2\sigma^{2} }}} \right), $$
(6)

where C 1 is a constant.

The PDF p(z) is the image prior, which imposes the spatial constraints on the image. For the image prior term, the Laplacian prior [17, 39] and the Gauss–Markov prior [36] are commonly employed. A common criticism of these regularization methods is that detailed information in the estimates tends to be overly smoothed [32]. Therefore, an edge-preserving Huber–Markov image prior is employed in this paper. This prior can effectively preserve the edge and detailed information in the image [16]. The Huber–Markov prior has the following form:

$$ p({\mathbf{z}}) = \frac{1}{{C_{2} }}\exp \left( { - \frac{1}{{2\lambda_{1} }}\sum\limits_{x,y} {\sum\limits_{c \in C} {\rho (d_{c} ({\mathbf{z}}_{x,y} ))} } } \right). $$
(7)

In this expression, C 2 is a constant, c is a clique within the set of all image cliques C, and the quantity d c (z x,y ) is a spatial activity measure of pixel z x,y , which is often formed by first- or second-order differences. The potential function ρ(i) is the Huber function, defined as

$$ \rho (i) = \left\{ {\begin{array}{ll} {i^{2} } & { |i|\le \mu } \\ {2\mu |i| - \mu^{2} } & { |i| > \mu } \\ \end{array} } \right. $$
(8)

where μ is a threshold parameter separating the quadratic and linear regions [16]. As for the function d c (z x,y ), we compute the following finite second-order differences in four adjacent cliques for every location (x, y) in the SR image:

$$ \begin{aligned} d_{c}^{1} ({\mathbf{z}}{}_{x,y})& = {\mathbf{z}}{}_{x - 1,y} - 2{\mathbf{z}}_{x,y} + {\mathbf{z}}_{x + 1,y} \hfill \\ d_{c}^{2} ({\mathbf{z}}_{x,y} ) &= {\mathbf{z}}_{x,y - 1} - 2{\mathbf{z}}_{x,y} + {\mathbf{z}}_{x,y + 1} \hfill \\ d_{c}^{3} ({\mathbf{z}}_{x,y} ) &= \frac{1}{\sqrt 2 }\left[ {{\mathbf{z}}_{x - 1,y - 1} - 2{\mathbf{z}}_{x,y} + {\mathbf{z}}_{x + 1,y + 1} } \right] \hfill \\ d_{c}^{4} ({\mathbf{z}}_{x,y} ) &= \frac{1}{\sqrt 2 }\left[ {{\mathbf{z}}_{x - 1,y + 1} - 2{\mathbf{z}}_{x,y} + {\mathbf{z}}_{x + 1,y - 1} } \right] \hfill \\ \end{aligned}.$$
(9)

The PDF p(b) is the blur function prior, which imposes the spatial constraints in the blur domain. As the blurring function is unknown, we should take the piecewise smoothness property of the blurring function into consideration. In this paper, the following Laplacian prior model [25, 27] is used:

$$ \, p({\mathbf{b}}) = \frac{1}{{C_{3} }}\exp \left( { - \lambda_{2} \left\| {{\mathbf{Qb}}} \right\|^{ 2} } \right),$$
(10)

where λ 2 is a parameter that controls the contribution of the blurring function prior model, and Q is a 2-D Laplacian here.

The choice of a prior model for the registration parameters is highly application specific [36]. If there are relatively few registration parameters to estimate, a “no preference” prior can yield a useful solution. Considering that a translational motion model is used in this paper, there are only two registration parameters for each LR image, and a “no preference” prior is used here. Thus, the estimation of the registration parameters reduces to an ML estimate [36].

Substituting (6), (7), and (10) into (5), after some manipulation, the minimization cost function is obtained in (11)

$$ {\hat{\mathbf{z}}},{\hat{\mathbf{b}}},{\hat{\mathbf{s}}} = \arg \hbox{min} \left\{ {\left\| {{\mathbf{g}} - {\mathbf{DM}}(s){\mathbf{Bz}}} \right\|^{2} + \alpha \sum\limits_{x,y} {\sum\limits_{c \in C} {\rho (d_{c} ({\mathbf{z}}_{x,y} ))} } + \beta \left\| {{\mathbf{Qb}}} \right\|^{2} } \right\}.$$
(11)

This cost function is used in the optimization procedure introduced subsequently.

4 Optimization Procedure

It is noted that the cost function (11), as a joint function of three sets of variables, is not convex. Therefore, a cyclic coordinate descent optimization procedure is developed to solve the unknowns. The registration parameters, blurring function, and HR image are found in an alternate manner, given the two others, respectively.

4.1 Updating Unknowns

Given the full set of registration parameters s and blurring functions b, the desired HR image z can be updated by minimizing the following cost function:

$$ E_{1} ({\mathbf{z}}) = \left\| {{\mathbf{g}} - {\mathbf{DM}}(s){\mathbf{Bz}}} \right\|^{2} + \alpha \sum\limits_{x,y} {\sum\limits_{c \in C} {\rho (d_{c} ({\mathbf{z}}_{x,y} ))} }, $$
(12)

which is composed of the two terms in (11) that contain the sole unknown quantity z. This is a regularized optimization problem. To solve the desired HR image z, we use the method of conjugate gradients.

Given the estimate of HR image z and registration information s, the cost function to estimate the blurring function b is given by:

$$ E_{2} ({\mathbf{b}}) = \left\| {{\mathbf{g}} - {\mathbf{DM}}(s){\mathbf{Zb}}} \right\|^{2} + \beta \left\| {{\mathbf{Qb}}} \right\|^{2}. $$
(13)

It is common to assume that the blurring function is positive and preserves image brightness, so the blurring function should satisfy the following constraints:

$$ {\mathbf{b}}_{k} (i,j) \ge 0\quad {\text{ and }}\quad \sum\nolimits_{i,j} {{\mathbf{b}}_{k} (i,j)} = 1. $$
(14)

Therefore, we can restrict the intensity values of the blurring function between 0 and 1. In order to enforce these bounds, we solve (13) as a constrained optimization problem. Constrained optimization problems are more computationally demanding, but we can afford it in this case since the size of the blurring function is much smaller than the size of HR image.

Given the estimate of the HR image z and blurring function b, the cost function to estimate the registration information s can be given as

$$ E_{3} ({\mathbf{s}}) = \left\| {{\mathbf{g}} - {\mathbf{DM}}(s){\mathbf{Bz}}} \right\|^{2}. $$
(15)

A search is required to minimize (15) with respect to s. Pyramidal search strategies designed for traditional block matching can be employed here [36]. As opposed to the two-stage blind SR methods that perform registration on the LR images in the first stage, the image registration is performed iteratively here, using the progressively estimated HR image and blurring function. It is believed that more precise registration parameters can be obtained, thereby enhancing the performance of the HR reconstruction and blur identification.

4.2 Initialization

Before updating all these unknowns, the HR image, registration information, and blurring function must be initiated. The initial HR image is obtained using bicubic interpolation of the reference LR image. The registration parameters are initiated by implementing an existing registration algorithm, such as the parameter model-based image registration algorithm [32]. The initial estimate of the blurring function is obtained using the method described in [27], which should not be far away from the real values. The two regularization parameters in (11) are determined heuristically, according to quantitative evaluations or visual assessments in our experiments. That is, a set of properly chosen parameter values are tested, and the values which correspond to the optimal quantitative evaluations and visual assessments are selected.

A block diagram of the whole optimization procedure is shown in Fig. 2. The number of iteration (N = 10) is used as a stopping criterion in this paper.

Fig. 2
figure 2

Diagram of the joint MAP algorithm

Full size image

5 Experimental Results

In this section, we illustrate the performance of the proposed method on two different sets of data: simulated and real-life images. To evaluate the performance, the normalized mean-square error (NMSE) and peak signal-to-noise ratio (PSNR) [42, 43] were employed as quantitative measures for the estimated registration information, blurring function, and HR image in the simulated experiments. Generally speaking, a good algorithm is reflected by low NMSE and high PSNR.

5.1 Simulated Images

We conducted simulation experiments to perform blind SR on multiple noisy LR images with different blur levels. The “cameraman” image in Fig. 3a was selected as the test image. To generate the LR images, the HR image was first shifted with 0, 1, 2, or 3 HR pixels in the horizontal and vertical directions, respectively, and then blurred with a symmetric Gaussian blur before downsampling by a decimation factor of 4. The LR images were further degraded by AWGN, yielding a signal-to-noise ratio (SNR) of 30 dB. We experiment with Gaussian blurs of variances 0.75, 1.0,, and 1.5 and try to estimate the blurring functions, assuming that the support of the PSF has been known. For each level of Gaussian blur, 16 synthetic degraded images are generated and used by a resolution enhancement factor of 4 in this experiment. The two-stage disjoint blind image SR method, which doesn’t consider image registration errors, was utilized as the benchmark algorithm of the proposed method. In addition, the image SR reconstruction with the exactly known blur and registration information, was also applied to the observed LR images to give full comparisons with the experimental results of the proposed method. The two-stage disjoint blind SR method was implemented with two steps of independent image registration procedure over the LR observed images and solving the MAP cost function in (11) with respect to the HR image and blurring function, iteratively, which is very similar to the methods in [20], except for the types of blurring function and image priors utilized to ensure the fairness of comparisons. Our proposed method was run based on the procedure shown in Fig. 2. We performed image registration using the translational parameter model-based image registration algorithm, which is a special case of the six-parameter affine model in [32]. SR reconstruction with known blur and registration information was implemented by minimizing the cost function in (12), assuming that the blur and registration information were exactly known.

Fig. 3
figure 3

SR reconstruction result of LR images with Gaussian blur of variance 0.75. a Original HR image, b the referenced LR image, c bicubic interpolation, d two-stage disjoint blind SR method without considering image registration errors, and e proposed method and f SR reconstruction result with exactly known blur and registration information

Full size image

The NMSE of the registration information under different Gaussian blur levels is given in Table 1. Comparing the image registration information obtained with predefined image registration method [32], the proposed method updates the registration information using the progressively estimated blurring function with the HR image and greatly improves image registration accuracy. Tables 2 and 3, respectively, show the NMSE of the estimated blurring function and PSNR of the reconstructed HR images under different Gaussian blur levels. From these tables, it is clearly observed that the proposed method achieves a more accurate blurring function and better reconstruction results than the two-stage disjoint blind SR method without considering image registration errors. These observations further reconfirm the NMSE results of the registration information obtained with the proposed method with considering image registration errors in the blurring information estimation and image reconstruction processes from another perspective. Comparing our results with those of the two-stage disjoint blind SR method, it is found that the improvement of image registration has a direct effect on the blur identification and SR reconstruction, thereby improving the accuracy of estimated blur function and the quality of the reconstructed HR image.

Table 1 NMSE of the registration information under different Gaussian blur levels
Full size table
Table 2 NMSE of the blurring function under different Gaussian blur levels
Full size table
Table 3 PSNR of the reconstructed HR image under different Gaussian blur levels
Full size table

Now the emphasis is on the evaluation and analysis of the reconstructed HR images. Here we compared the reconstructed HR image results of the three sets of LR images, to validate the performance of the proposed method under different Gaussian blur levels. The threshold parameter μ in (8) is set as 5 in all three experiments. The SR reconstruction results with Gaussian blurs of variances 0.75, 1.0, and 1.5 are illustrated in Figs. 3, 5, and 7, respectively. To facilitate a better comparison, a region of each is shown in detail in Figs. 4, 6, and 8, respectively. Comparing the reconstruction results in Figs. 3, 4, 5, 6, 7, and 8, it is obviously observed that the image reconstructed by the two-stage disjoint blind SR method has much better visual quality than the interpolated image. However, artifacts are displayed around the edges because of the existence of the image registration errors, such as at the right elbow of the cameraman. The proposed method makes an obvious improvement best described as a suppression of these artifacts by considering the image registration errors in the blurring information estimation and image reconstruction processes, and obtains a very close result to the SR reconstruction result with exactly known blur and registration information. From the quantitative comparison results given in Table 3, it is observed that quantitative measures agree with the visual evaluation. Evidently, the proposed algorithm outperforms the two-stage disjoint blind SR algorithm, in terms of both the quantitative measurements and visual evaluation, which validate the effectiveness of considering the image registration errors within the proposed image blind SR reconstruction procedure.

Fig. 4
figure 4

a–e Sampled regions cropped from Fig. 3(a), (c)–(f), respectively

Full size image
Fig. 5
figure 5

SR reconstruction result of LR images with Gaussian blur of variance 1.0. a Original HR image, b the referenced LR image, c bicubic interpolation, d two-stage disjoint blind SR method without considering image registration errors, and e proposed method and f SR reconstruction result with exactly known blur and registration information

Full size image
Fig. 6
figure 6

a–e Sampled regions cropped from Fig. 5(a), (c)–(f), respectively

Full size image
Fig. 7
figure 7

SR reconstruction result of LR images with Gaussian blur of variance 1.5. a Original HR image, b the referenced LR image, c bicubic interpolation, d two-stage disjoint blind SR method without considering image registration errors, and e proposed method and f SR reconstruction result with exactly known blur and registration information

Full size image
Fig. 8
figure 8

a–e Sampled regions cropped from Fig. 7(a), (c)–(f), respectively

Full size image

5.2 Real-life Images

The real-life images experiment was conducted on the “castle” image sequence. We used four images with image 1 as the referenced image, which is shown in Fig. 9a. The parameter model-based image registration algorithm in [32] was again used to estimate the registration parameters from the LR images. Next, the two-stage disjoint blind SR method without considering image registration errors and our proposed method were run to perform blind image SR reconstruction. Figure 9b–d shows the bicubic interpolated image, the two-stage disjoint blind SR result and the result by the proposed method, respectively. Sampled regions cropped from Fig. 9b–d are depicted in Fig. 10a–c, respectively. The parameters used for this example are set as follows: α = 0.01, β = 10,000, and μ = 5. From the figures, it is observed that the considerable clarity of the images has been recovered by both the two-stage disjoint blind SR method and the proposed method. Further, it can be seen that the result by our proposed method has fewer artifacts than in the two-stage disjoint blind SR method, in particular near the edges. Comparison reveals that our approach is superior in handling real-life blind image SR, as it is able to estimate image registration parameters accurately, leading to superior blur identification and HR image reconstruction.

Fig. 9
figure 9

Blind SR reconstruction of the “castle” image sequence. a The referenced LR image, b bicubic interpolation, c two-stage disjoint blind SR method without considering image registration errors, and d the proposed method

Full size image
Fig. 10
figure 10

a–c Sampled regions cropped from Fig. 13(b)–(d), respectively

Full size image

6 Conclusion

This paper presents a new blind SR reconstruction method by integrating image registration into blind image SR reconstruction to reduce the effect of registration errors, and then a cyclic coordinate descent optimization procedure is developed to solve the formulation. As opposed to the two-stage blind SR methods that perform image registration and blind SR reconstruction as disjoint processes, the new framework enables image registration, blur identification, and HR image reconstruction to be estimated and improved progressively. The advantage of this algorithm is that image registration information, blurring function, and the HR image can benefit each other. The proposed algorithm was tested on simulated, as well as real-life images. The experimental results validated that registration parameters, blurring information, and the HR image can be noticeably improved by implementing this algorithm. Nevertheless, there may still be room for improvements to our optimal method, to increase computational efficiency. Using more robust prior models for the blurring function (e.g., the best-fit parametric blur model [22]) may further improve the SR results. These will be addressed in our future work.