1 Introduction

Image interpolation is an essential tool for image processing scientists to acquire a HR image from a LR one. The motivation for the research in this field of image processing is the inability of most imaging sensors to attain the required HR image at a moderate cost [2, 14]. The application of image interpolation in the field of IR image processing is a promising trend, as IR images usually have low resolutions [15, 23]. Traditional kernel-based image interpolation algorithms have been broadly studied. Most of the exploration in these kernel-based techniques was to obtain the best interpolation basis functions [9, 21, 22, 30].

Splines, keys and o-moms are the most common families utilized to deduce the best interpolation basis functions. These conventional methods are space-invariant and do not consider the spatial activity of the picture to be interpolated [25, 30, 35]. They also are not the mathematical model of the imaging process with a specific type of sensors. Spatially-adaptive kernel depended on techniques, for example the warped-distance method, have been developed [6, 29]. Although these adaptive methods improve the quality of the interpolated image, especially, near edges, they still do not take into consideration the image capturing mathematical model. The entire kernel-based algorithms and their adaptive variants can be considered as signal synthesis algorithms [3, 6].

This paper drives its motivation and importance through the research topic and nature of images used. The rapid and massively increasing development of Image processing technologies makes the acquiring of SR image is a research hotspot with a very wide range of applications [5, 17] moreover applying SR techniques on a difficult nature images like IR images makes it very challenging. The main challenge of acquiring a HR image directly through an IR camera is the manufacturing difficulty, materials properties and imaging environment. Although some researches offers designs to enhance the scanning parts such as [4, 12, 24]. The method that offered the enhancement based on IR optical system by using four different angles plate refractors placed in parallel inside the device is shown in [7, 8, 13]. It stills application limited due to fabrication complexity, size, cost and other manufacturing challenges [20, 26].

Those researches drives to acquire HR IR image from a one or more LR images using HR techniques [27, 32] .Offering three categories techniques based on image processing technologies interpolation [4, 5, 17], reconstruction [4, 12, 24] and learning. Interpolation techniques are the earliest and the most essential tool in image processing although the reconstructed image don’t satisfy the required quality, while reconstruction techniques offers a pretty stage in image enhancement that assumes the LR image is a result of degradation models such as noise, distortion, blurring and down sampling but still the need of prior knowledge to accurately obtain HR images from LR one. Learning techniques represents the new era in image processing resulting a satisfying results without the need of prior knowledge as it gets the needed knowledge by studying the relationship between LR and HR images through a pre-learned dataset of LR and HR image patches which means more processing at the initial setup in the learning stage [27, 32, 33].

Applying SR techniques on IR images is a very interesting newly growing research topic that researchers are offering different methods most are based on learning techniques in different ways [19, 34]. One of the methods [27] offers the learning stage through combining the information from visible images and IR images as images from different sensors carries complementary information for the same scene, another method [33] offers the reconstruction of SR image based on the sparse representation of images offering a pair of dictionaries where HR and LR patches share the same sparse representation, another add on is the using of compressed sensing (CS) [10, 11, 19, 34] making benefits of working with the CS framework [1, 16, 18, 28, 31] to solve the sparsity reconstruction problem.

This research submits three novel models for IR images. The first technique is based on image SR techniques that are used for resolution enhancement of LR images by estimating an HR image from one or more LR images. This suggested technique used to give more details and enhanced visibility with SR image from LR one provides a pure big data processing model with respect to the data size nowadays. The second scheme is based on interpolation for IR images using SKLI. The third scheme is depended on up-sampling for IR images using TKLI.

The organization of the paper as follows: Section 2 gives motivations and related work. Section 3 explains acquisition methods. Section 4 explains REI. Section 5 explains LSI of IR images. Section 6 explains the SR applied to IR images. Section 7 presents the first presented scheme. Section 8 presents the second presented scheme. Section 9 presents the third presented scheme. Section 10 gives the simulation results. Finally, Section 11 gives the concluding remarks.

2 Motivations and related work

The rapid and massively increasing development of image processing technologies makes the acquisition of SR images a hot research topic with a very wide range of applications [3, 6, 17, 29]. Moreover, applying SR techniques on low-quality images like IR images makes it very challenging. The main limitation to acquiring an HR image directly through an IR camera is the manufacturing difficulty, materials properties, and imaging environment. Some researches offered designs to enhance the scanning section of the IR imaging system [4, 5, 8, 12, 13, 24]. These designs offered enhancement schemes using four different-angle plate refractors placed in parallel inside the device. This design is still application limited due to fabrication complexity, size, and cost.

The manufacturing challenges directed research towards the acquisition of HR IR images from one or more LR images using SR techniques [7, 20, 26, 27, 32]. The SR acquisition techniques can be classified into three categories; interpolation techniques, reconstruction techniques and learning techniques. Interpolation techniques are the earliest and the most essential methodologies adopted for SR acquisition, but they give images with low quality [4]. On the other hand, reconstruction techniques provide better-quality images by assuming that the LR image is a result of a degradation model comprising noise, distortion, blurring, and down-sampling, but there is still a need for prior knowledge to accurately obtain HR images from the LR ones. Learning techniques represent a new era in image processing resulting in satisfactory results without the need for prior knowledge about the LR degradation model. So, the concepts of learning are adopted in this paper [19, 33, 34].

Applying SR techniques on IR images is a very interesting and growing research topic, and researchers offer different methods for SR acquisition from IR images based on learning techniques [19, 27, 33, 34]. One of the methods [27] offers the learning stage through combining the information from visible and IR images as images from different sensors carrying complementary information for the same scene. Another method [33] presents the reconstruction of SR images based on the sparse representation of them by offering a pair of dictionaries, in which the HR and LR patches share the same sparse representations. Another trend adopts compressed sensing (CS) [19, 34] for SR acquisition of images making benefits of working with the CS framework [1, 10, 11, 16, 18, 28, 31] to solve the sparsity reconstruction problem acquired from LR sensors, application such as synthetic aperture radar, or IR imaging, or due to data storage limitations.

3 Acquisition methods

In this paper, we will concentrate on how to perform the enhancement of IR images using SR techniques. We propose different techniques to perform the required enhancement. The main objective is to process the LR IR images to obtain HR images, which are more suitable than the original ones for specific applications. Image enhancement is one of the most appealing areas of image processing that witnessed a significant progress and a remarkable development.

The methods of acquisition of SR images are classified into two main categories as shown in Fig. 1: classical or conventional SR methods [19, 33, 34] and SIMSR methods.

Fig. 1
figure 1

Acquisition methods of SR images

Full size image

Classical or conventional SR methods, as shown in Fig. 2a, are based on combining multiple LR images of the same scene obtained at sub-pixel misalignments. On the other hand, SIMSR methods, as shown in Fig. 2b, require only one LR image for the prediction and construction of the SR image making benefits of a pre-learned database for the correspondence between the LR image patches and the HR ones.

Fig. 2
figure 2

Categories of different SR methods (a) Classical or conventional SR methods (b) SIMSR methods

Full size image

The selection of a suitable image enhancement technique is based on the application itself, as there is no absolute theory or ultimate concept of image enhancement. A certain amount of trials and errors is usually required to get the most appropriate enhancement technique. When an image is processed for visual inspection, only the viewer is the ultimate evaluator for the performance of the used enhancement algorithm, method or technique. Visual evaluation of image quality is considered as a subjective process, and thus it makes the definition of a good image a devious standard. When images are being processed for a certain application perception, the evaluation task becomes much easier. The best image processing method would be the one yielding the best results for the application.

An SIMSR algorithm for enhancement of IR images is proposed in this chapter. The proposed algorithm relies on the image acquisition model, which considers benefits of the sparse representations of LR and HR patches of the images. It uses Bi-cubic interpolation and Minimum Mean Square Error (MMSE) estimation in the prediction of the HR image providing a scheme that can be interpreted as a feed-forward neural network.

The proposed algorithm overcomes the limitation of having only LR images due to hardware limitations. It is represented with a big data processing model. The performance of the suggested algorithm is compared with the standard regularized image interpolation technique as well as an adaptive block-by-block Least Squares (LS) interpolation technique from the Peak Signal-to-Noise Ratio (PSNR) perspective. It considers both visual quality and resolution enhancement of the IR images. Results reveal the superiority of the proposed SR algorithm.

4 REI

This method can be formulated as the constrained minimization of a certain functional, called stabilizing functional. The specific constraints imposed by this approach on the solution that depend on the properties and form of the stabilizing functional used [11].

The solution of the REI problem is obtained by the minimization of the cost function [11]:

$$ \boldsymbol{\Psi} \left(\hat{\mathbf{f}}\right)=\left\Vert \mathbf{g}-\mathbf{D}\hat{\mathbf{f}}\right\Vert +\lambda {\left\Vert \mathbf{Q}\hat{\mathbf{f}}\right\Vert}^2 $$
(1)

where D is the decimation operator that is assume to relate the HR to the LR image, g is LR image,\( \hat{\mathbf{f}} \) is the estimated HR image, Q is the regularization operator, and λ is the regularization parameter.

This minimization is created by taking the derivative of the cost function yielding:

$$ \frac{\mathrm{\partial \Psi}\left(\hat{\mathbf{f}}\right)}{\partial \hat{\mathbf{f}}}=\mathbf{0}=2{\mathbf{D}}^T\left(\mathbf{g}-\mathbf{D}\hat{\mathbf{f}}\right)-2\lambda {\mathbf{Q}}^T\mathbf{Q}\hat{\mathbf{f}} $$
(2)
$$ 2\hat{\mathbf{f}}\left({\mathbf{D}}^T\mathbf{D}+{\uplambda \mathbf{Q}}^T\mathbf{Q}\right)-2{\mathbf{D}}^T\mathbf{g}=\mathbf{0} $$
(3)

where T refers to matrix transpose.

Solving for \( \hat{\mathbf{f}} \) that provides the minimum of the cost function yields [11]:

$$ \hat{\mathbf{f}}={\left({\mathbf{D}}^T\mathbf{D}+{\lambda \mathbf{Q}}^T\mathbf{Q}\right)}^{-1}{\mathbf{D}}^T\mathbf{g}=\mathbf{A}\left(\lambda \right)\mathbf{g} $$
(4)

The rule of the Q is to move the small eigenvalues of D away from zero, while leaving the large eigenvalues unchanged. It also incorporates prior knowledge about the required degree of smoothness of \( \hat{\mathbf{f}} \) into the interpolation process.

The generality of the linear operator Q allows the development of a variety of constraints that can be incorporated into the interpolation operation. For instance [11]:

1- Q = I. In this case the regularized solution reduces to the pseudo-inverse filter solution, and it is represented as:

$$ \hat{\mathbf{f}}={\left({\mathbf{D}}^T\mathbf{D}+\uplambda \mathbf{I}\right)}^{-1}{\mathbf{D}}^T\mathbf{g} $$
(5)

2- Q = finite difference matrix. In this case, the operator Q is chosen to minimize the second order difference energy of the estimated image.

The 2-D Laplacian is preferred for minimizing the second-order difference energy. It is the most popular regularization operator. The λ controls the trade-off between the fidelity of the data and the smoothness of the solution.

The solution of REI problem is implemented by the segmentation of LR image into overlapping segments and the interpolation of each segment separately using Eq. (6) .The REI formula can be given as the following:

$$ {\hat{\mathbf{f}}}_{i,j}={\left({\mathbf{D}}^T\mathbf{D}+{\lambda \mathbf{Q}}^T\mathbf{Q}\right)}^{-1}{\mathbf{D}}^T{\mathbf{g}}_{i,j} $$
(6)

where gi, j and \( {\hat{\mathbf{f}}}_{i,j} \) are the lexicographically-ordered LR and the estimated HR blocks at position (i, j), respectively.

5 LSI

In this technique, the IR image to be interpolated is split into small overlapping blocks to obtain an interpolated version of each block. It is supposed that the relationship between the available LR and the estimated HR block is offered by [4]:

$$ {\hat{\mathbf{x}}}_{ij}={\mathbf{WY}}_{ij} $$
(7)

where Yi, j and\( \kern0.5em {\hat{\mathbf{X}}}_{i,j} \) are the lexicographically-ordered LR and the estimated HR blocks at the block indices (i, j), respectively. W is the weight matrix required to obtain the HR block from the LR block. This matrix is required to be adaptive from block to block to accommodate for the local activity levels of each block.

The first look at Eq. (6) leads to the LS solution that can be obtained by minimizing the Mean Square Error (MSE) of the estimation as follows [4]:

$$ \boldsymbol{\Psi} ={\left\Vert {\mathbf{X}}_{ij}-{\hat{\mathbf{X}}}_{ij}\right\Vert}^2={\left\Vert {\mathbf{X}}_{ij}-{\mathbf{WY}}_{ij}\right\Vert}^2 $$
(8)

Differentiating both sides of Eq. (13) with respect to W gives:

$$ \frac{\partial \boldsymbol{\Psi}}{\partial \mathbf{W}}=-2\left({\mathbf{X}}_{ij}-{\hat{\mathbf{X}}}_{ij}\right){\left({\mathbf{Y}}_{ij}\right)}^T $$
(9)

This minimization leads directly to the following solution for W.

$$ {\mathbf{W}}^{k+1}={\mathbf{W}}^k-\eta {\left[\frac{\partial \boldsymbol{\Psi}}{\partial \mathbf{W}}\right]}^k={\mathbf{W}}^k+\mu \left({\mathbf{X}}_{ij}-{\hat{\mathbf{X}}}_{ij}^k\right){\left({\mathbf{Y}}_{ij}^k\right)}^T $$
(10)

where η is a constant and μ is the convergence parameter and k  is the iteration number.

So, we consider that the model relates the available LR block to the original HR block, illustrated in Fig. 3. This model is offered by the following relation [16]:

$$ {\mathbf{Y}}_{i,j}={\mathbf{HX}}_{i,j} $$
(11)
Fig. 3
figure 3

LSI from an N × N HR block to an N/2 × N/2 LR block

Full size image

The matrix H represents the filtering and down-sampling process that transforms the HR block to an LR block.

Thus, we can get the following cost function:

$$ \boldsymbol{\Phi} ={\left\Vert \mathbf{H}\left({\mathbf{X}}_{\mathbf{i},\mathbf{j}}-{\hat{\mathbf{X}}}_{\mathbf{i},\mathbf{j}}\right)\right\Vert}^2 $$
(12)

The above equation means the MSE between the available LR block and a down-sampled version of the estimated HR block.

This leads to:

$$ \boldsymbol{\Phi} ={\left\Vert {\mathbf{Y}}_{\mathbf{i},\mathbf{j}}-\mathbf{H}{\hat{\mathbf{X}}}_{\mathbf{i},\mathbf{j}}\right\Vert}^2={\left\Vert {\mathbf{Y}}_{\mathbf{i},\mathbf{j}}-{\mathbf{HWY}}_{\mathbf{i},\mathbf{j}}\right\Vert}^2 $$
(13)

The LSI final Eq. [6]:

$$ {\mathbf{W}}^{k+1}={\mathbf{W}}^k-\eta {\left[\frac{\partial \boldsymbol{\Psi}}{\partial \mathbf{W}}\right]}^k={\mathbf{W}}^k+{\mu \mathbf{H}}^T\left({\mathbf{Y}}_{ij}-\mathbf{H}{\hat{\mathbf{X}}}_{ij}^k\right){\left({\mathbf{Y}}_{ij}^k\right)}^T $$
(14)

The adaptation of Eq. (20) can be easily performed since it does not require the original HR block to be known a priori.

6 SR Applied to IR Images

This idea is based on applying an SR technique on IR images. The representation of LR and HR images as vectors are \( {\mathbf{z}}_l\in {\mathbf{R}}^{M_l} \) and \( {\mathbf{y}}_h\in {\mathbf{R}}^{M_h} \) respectively, where Mh = q2Ml and q is a scale-up factor of type integer that is higher than 1. We also refer to \( \mathbf{H}\in {\mathbf{R}}^{M_h\times {M}_h} \) as blur operator, and \( \mathbf{K}\in {\mathbf{R}}^{M_l\times {M}_h} \) as the decimation operator for q in each axis. A well-known anti-aliasing low-pass filter is applied on the image to generate an LR image from an HR image [18]:

$$ {\mathbf{z}}_l=\mathbf{K}{\mathbf{Hy}}_h+\mathbf{A} $$
(15)

where A is an additive noise vector or error in this process.

The relative problem of the reconstruction of yh from zl is denoted as zooming and deblurring. A bicubic low-pass filter and a Gaussian low-pass filter are possible blur kernels (blur operator H).

As the work is based on the sparse representation of patch pairs, we have to learn the correspondence between LR and HR patches of the same dimensions. So, we obtain an image \( {\mathbf{y}}_l\in {\mathbf{R}}^{M_h} \) by applying bicubic interpolation on the input LR image and as we address the zooming and deblurring setup, we look forward at recovering the difference image \( {\hat{\mathbf{y}}}_{hl}={\mathbf{y}}_h-{\mathbf{y}}_l \) and then apply \( {\hat{\mathbf{y}}}_h={\hat{\mathbf{y}}}_{hl}+{\mathbf{y}}_l \)to get the final recovery. So, we keep the LR details and only predict the lost HR details. For the concept of image reconstruction based on patches, let an image patch that is centred at location Q and of size \( \sqrt{m}\times \sqrt{m} \) and that is extracted from the image vector y of size Mh by the linear operator RQ to be  pQ = RQy . A local model could be suggested to predict an HR patch \( {\mathbf{P}}_h^Q \)=RQyh from an LR one \( {\mathbf{P}}_l^Q={\mathbf{R}}_Q{\mathbf{y}}_l \) . Once obtaining all HR patch predictions, the recovery of the HR image takes place by averaging of the overlapping recovered patches on their overlaps. Another factor should be taken into consideration is the trade-off between reconstructed image quality and the run time, while choosing the size of the overlap between adjacent patches. In order to achieve the best reconstruction quality, work has to be performed with maximally-overlapping patches (overlapping between adjacent patches is of \( \sqrt{m}\times \sqrt{m-1} \)pixels in the horizontal and vertical directions).

Finally, we briefly mention the sparsity-based synthesis model. The core idea of this model is that a signal S ∈ Rm can be represented as a linear combination of signal prototypes taken from a dictionary D ∈ Rm × n , namely  S = Dα + η, where α ∈ Rnis the sparse representation vector and η is noise or model error. Similar to the previous approaches [35], we assume that each LR patch can be represented over a dictionary \( {\mathbf{D}}_l\in {\mathbf{R}}^{m\times {n}_l}\kern0.5em \)by a sparse vector\( \kern0.75em {\boldsymbol{\upalpha}}_l\in {\mathrm{R}}^{n_l} \), and similarly a HR patch is represented over \( {\mathbf{D}}_h\in {\mathbf{R}}^{m\times {n}_h} \) by \( {\boldsymbol{\upalpha}}_h\in {\mathbf{R}}^{n_h} \) [16, 20].

6.1 Low-cost pursuit stage

To sparsely represent the patches, an under-complete dictionary (nl < m) is sufficient enough. To allow a low-cost scale-up scheme Dlis assumed as an under-complete orthonormal dictionary. Therefore, the inner products of the LR patch with the dictionary atoms are shown in Fig. 4,the LR coefficients results.

$$ {\boldsymbol{\upalpha}}_l={\left({\mathbf{D}}_l\right)}^T{\mathbf{P}}_l $$
(16)
Fig. 4
figure 4

Patching Process

Full size image

A convolutional network is then used to compute the LR coefficients for all overlapping patches\( \left\{{\mathbf{P}}_l^Q\right\} \). The sparsity pattern \( {\mathbf{x}}_l\in {\left\{-1,1\right\}}^{n_l} \) is computed as:

$$ {x}_{l,j}=\left\{\begin{array}{c}1,\kern0.5em \left|{\upalpha}_{l,j}\right|>\delta \\ {}-1,\kern2em Otherwise\end{array}\kern0.5em ,\kern1.75em {\forall}_j=1,\dots, {n}_{l,}\right. $$
(17)

where δ is the maximal threshold satisfying that set, adaptively, for each LR patch based on a residual error criterion.

$$ \sum \limits_{j=1}^{n_l}{\left|{\upalpha}_{l,j}\right|}^2\left(\left|{\upalpha}_{l,j}\right|\le \updelta \right)\le m{\rho}^2 $$
(18)

Where, ρ is a pre-specified parameter that indicates the targeted accuracy.

6.2 Neural network model

The proposed model for single-image SR can be interpreted as a feed-forward neural network providing a highly fast and simple implementation. The objective of this proposed model is to find the network parameters to get the best prediction of HR patches from the corresponding LR ones. The suggested network consists of the following parameters,

$$ \Theta =\left\{{\mathbf{D}}_l,{\mathbf{D}}_h,{\mathbf{S}}_{hl},{\mathbf{b}}_h,{\mathbf{V}}_{hl}\right\} $$
(19)

We assume that the process of learning the model parameters is off-line, using a set of LR-HR image pairs. Patches are extracted from each image pair yl, yhl at the same locations, resulting in a training set consisting of N paired LR HR patches\( \kern0.5em \left\{{\mathbf{P}}_l^Q,{\mathbf{P}}_h^Q\right\} \).

The optimization problem formulates the training model parameters  Θ [13]

$$ \frac{Argmin}{\Theta}\sum \limits_{K=1}^N{\left\Vert {\mathbf{D}}_h\left(\left[\Phi \left({\mathbf{b}}_h+{\mathbf{V}}_h{\mathbf{x}}_l^Q\right)\right]\bigcirc \left[{\mathbf{S}}_{hl}{\left({\mathbf{D}}_l\right)}^T{\mathbf{P}}_l^Q\right]\right)-{\mathbf{P}}_h^Q\right\Vert}_2^2 $$
(20)

The product above is the Hadamard product.

To reduce the complexity of solving this joint optimization problem in order to allow learning of model parameters, an initial estimation of Dl, and Dh dictionaries is set using directional PCAs [28] and K-SVD [1] as well-known approaches. Having the true sparsity patterns for each patch pair and given the Dl, and Dh dictionaries estimates, setting an initial estimate for the covariance matrix Shl directly by solving a LS problem. After having the mentioned initial estimates, Dh and Shl can be updated together to be well-tuned .Now, we reach the network innermost layer, where we update the restricted Boltzmann machine parameters Vhl , bh, while the remaining parameters are kept fixed to the previous estimates. Finally, a last tuning of the Dh dictionary takes place to enhance the prediction in terms of HR patch error.

7 The first presented scheme

This approach is based on single-image super resolution applied to IR images. This scheme is shown in Fig. 5.

Fig. 5
figure 5

Block diagram of proposed SIMSR applied to IR Images

Full size image
Fig. 6
figure 6

Block diagram of the first proposed algorithm

Full size image

The steps of this approach shown in Fig. 6 can be summarized as follows:

Fig. 7
figure 7

Case 1 of truck image SNR = 25 dB. a LR IR image b REI result c LSI result d The SR second scenario model result  e The SR third scenario model result g The second suggested scheme result h The third suggested scheme result

Full size image

Input: The LR IR image zl, Scale up factor q, and model parameters Dl, Dh, Shl, bh, Vhl .

Output: Estimated HR image yh.

Capture LR IR image zl.

Apply bicubic interpolation with scale up factor q on the zl to generateyl.

  1. 1.

    Extract overlapping patches\( \left\{{\mathbf{P}}_l^Q\right\} \) at centred location Q from yl image. .

  2. 2.

    Compute the LR representation αl from the LR patches Pl and Dl using Eq. (16).

  3. 3.

    Compute the LR sparsity pattern xl from the LR representation αl using Eq. (17).

  4. 4.

    Compute the MMSE for the HR representation αh using Eq. (20).

Apply the recovery process for the HR patches Ph from the LR representation and Dh .

Recover the HR imageyh.

8 The second presented scheme

This scheme presents up-sampling using SKLI. Lanczos is a spatial domain interpolation technique which is implemented by multiplying a sinc function with a sin window which is scaled to be wider and truncated to zero outside of the main lobe. In case of the SKLI, the main lobe of the sinc function along with the one subsequent side lobes on either side is used as a sinc window.

The Lanczos window is a product of sinc functions sinc(x)with the scaled version of the sinc function sinc(x / a) restricted to the main period −a ≤ x ≤ a to form a convolution kernel for re-sampling the input field [9]. In one dimension, the Lanczos interpolation formula is given by.

$$ L\left(\mathrm{x}\ \right)=\left\{\begin{array}{c}\ \\ {}\mathrm{sic}(x)\mathrm{sic}\left(\frac{x}{a}\right)\kern1.5em -a\le x\le a\\ {}0\kern4.75em otherwise\kern1em \\ {}\kern13em \\ {}\kern2.75em \end{array}\kern10.5em \right. $$
(21)

Equivalently,

$$ L\left(\mathrm{x}\ \right)=\left\{\begin{array}{c}\kern0.75em 1\kern10.75em x=0\\ {}\frac{\begin{array}{c}\kern0.5em \\ {}\mathrm{a}\ \sin \left(\pi x\right)\sin \left(\frac{\pi x}{a}\right)\end{array}}{{\left(\pi x\right)}^2}\kern5.25em -a\le x\le a\kern1em \\ {}\kern1.5em 0\kern9em otherwise\\ {}\kern2.25em \\ {}\kern0.5em \end{array}\right. $$
(20)

where a is the filter size parameter that determines the size of the kernel. The Lanczos kernel has (2a − 1) lobes: a positive one at the center, and (a − 1) alternating negative and positive lobes on each side, L(x) is the Lanczos kernel.

The two lobed Lanczos windowed sinc function for the SKLI is given by:

$$ {L}_2\left(\mathrm{x}\ \right)=\left\{\begin{array}{c}\ \\ {}\frac{\sin \left(\pi x\right)}{\pi x}\kern0.5em \frac{\sin \left(\frac{\pi x}{2}\right)}{\frac{\pi x}{2}}\kern5.25em -2\le x\le 2\kern1em \\ {}\kern1.5em 0\kern9em otherwise\\ {}\\ {}\kern0.5em \end{array}\right. $$
(23)

where a is a positive integer, typically 2, is used for controlling the size of the kernel. The parameter a corresponds to the number of lobes of the sinc.

9 The third presented scheme

This technique suggests interpolation using TKLI. In case of the TKLI, the main lobe of the sinc function along with the two subsequent side lobes on either side is used as a sinc window.

The three lobed Lanczos windowed sinc function for the TKLI is given by:

$$ {L}_3\left(\mathrm{x}\ \right)=\left\{\begin{array}{c}\ \\ {}\frac{\sin \left(\pi x\right)}{\pi x}\kern0.5em \frac{\sin \left(\frac{\pi x}{3}\right)}{\frac{\pi x}{3}}\kern5.25em -3\le x\le 3\kern1em \\ {}\kern1.5em 0\kern9em otherwise\\ {}\kern2.25em \\ {}\kern0.5em \end{array}\ \right. $$
(24)

The effect of each input sample on the interpolated values is defined by the Filter’s reconstruction Kernel called the Lanczos Kernel. It is the normalized sinc(x) function. Then multiplied this function by the Lanczos window. This window is the central lobe of horizontally stretched for the sinc(x/a).

  • Interpolation formula

Given a one-dimensional signal with samples si, for integer values of i, the value S(x) interpolated at an arbitrary real argument x is obtained by the discrete convolution of those samples with the Lanczos kernel:

$$ s(x)=\sum \limits_{i=\left\lfloor x\right\rfloor -a+1}^{\left\lfloor x\right\rfloor +a}\left\lfloor x\right\rfloor {s}_i\ L\left(x-1\right) $$
(25)

where a is the filter size parameter that determines the size of the kernel and ⌊x⌋ is the floor function. The bounds of this sum are such that the kernel is zero outside of them. The Lanczos kernel has (2a − 1) lobes: a positive one at the center, and (a − 1) alternating negative and positive lobes on each side, L(x) is the Lanczos kernel.

For a two dimensional function such as an image s (x, y), an interpolated value at an arbitrary point (x0, y0) using TKLI is given by

$$ s\hat{\mkern6mu} \left(x,y\right)=\sum \limits_{i=\left\lfloor x\right\rfloor -a+1\kern0.5em }^{\left\lfloor x\right\rfloor +a}\sum \limits_{j=\left\lfloor y\right\rfloor -a+1}^{\left\lfloor y\right\rfloor +a}s\left(i,j\right)\ L\left(x-i\right)\ L\left(y-j\right) $$
(26)

where sˆ(x, y)denotes the interpolated up-sampled image.

x, y represents spatial coordinates.

The final equation for IR image interpolation using SKLI, upon substituting a = 2 is given by:

$$ s\hat{\mkern6mu} \left(x,y\right)=\sum \limits_{i=\left\lfloor x\right\rfloor -1\kern0.5em }^{\left\lfloor x\right\rfloor +2}\sum \limits_{j=\left\lfloor y\right\rfloor -1}^{\left\lfloor y\right\rfloor +2}s\left(i,j\right)\ L\left(x-i\right)\ L\left(y-j\right) $$
(27)

where sˆ(x, y)denotes the interpolated up-sampled image.

x, y represents spatial coordinates.

The final equation for IR image interpolation using TKLI, upon substituting a = 3 is given by:

$$ s\hat{\mkern6mu} \left(x,y\right)=\sum \limits_{i=\left\lfloor x\right\rfloor -2\kern0.5em }^{\left\lfloor x\right\rfloor +3}\sum \limits_{j=\left\lfloor y\right\rfloor -2}^{\left\lfloor y\right\rfloor +3}s\left(i,j\right)\ L\left(x-i\right)\ L\left(y-j\right) $$
(28)

where sˆ(x, y)denotes the interpolated up-sampled image.

x, y represents spatial coordinates.

10 Simulation results

In this section, four IR images have been used to test the REI method, the LSI model, the SR algorithm, SKLI and TKLI techniques. Firstly, the model of image down-sampling process given in Fig. 3 is applied to the original images to yield the LR images down-sampled by a factor of two in both directions.

The REI method, the LSI model, the SR algorithm, SKLI and TKLI techniques are then tested on the obtained LR images.

Figure 7 gives the proposed approaches results of the first experiment for SNR = 25 dB. Fig. 7a shows the original IR LR image. Fig. 7b shows the enhanced image after REI. Fig. 7c shows the enhanced image after the LSI. Fig. 7d shows the enhanced image after SR second scenario model. Figure 7e shows The SR third scenario model. Figure 7f gives the first proposed approach result. Figure 7g shows the second proposed approach result. Figure 7h shows the third proposed approach result.

Fig. 8
figure 8

Case 2 of truck image, SNR = 25 dB. a LR IR image b REI result c LSI result d The SR second scenario model result e The SR third scenario model result f The first suggested approach g The second suggested scheme result h The third suggested scheme result

Full size image

Similar experiments have been carried out on other IR images and the results are given in Figs. 8, 9, 10. The characteristics of all IR images that utilized in experiments are shown in Table 1. The quality metrics that are used for evaluating this research are the PSNR and computation time. The higher the value of the PSNR, the better is the ability to enhancement IR images. The numerical results of cases for all presented techniques are shown in Tables 2 and 3.

Fig. 9
figure 9

Case 3 of truck image SNR = 25 dB. a LR IR image b REI result c LSI result d The SR second scenario model result e The SR third scenario model result f The first suggested Approach result g The second suggested scheme result h The third suggested scheme result

Full size image
Fig. 10
figure 10

Case 4 of truck image SNR = 25 dB. a LR IR image b REI result c LSI result d The SR second scenario model result e The SR third scenario model result f The first suggested approach g The second suggested scheme result h The third suggested scheme result

Full size image
Table 1 IR Images dataset properties
Full size table
Table 2 Numerical PSNR Results for all cases utilizing different schemes
Full size table
Table 3 Numerical computational time results for all cases using different algorithms
Full size table

The obtained results prove that the success of three suggested techniques in SR IR images. By comparing the three presented algorithms, it is clear that the third suggested technique gives superior results than other modes. This technique is the best algorithms from point views the PSNR and the computation time.

From the Fig. 11 clear that, It is clear that the second suggested technique gives superior than REI and LSI schemes from point views PSNR and computation time. On the other hand the third presented technique is the best algorithms from point views PSNR and computation time to other techniques.

Fig. 11
figure 11

Numerical results graph for all images using different techniques with respect to PSNR

Full size image

From Tables 2 and 3, the higher value of the PSNR, the better is the resolution of IR images. The lower value of the computation time, the ability to enhancement IR images.

From these tables, it is clear that the results are good from the visual, PSNR and computational time perspectives.

11 Conclusion

This paper presents three novel techniques for SR IR images. The first algorithm is based on picture acquisition model, that utilizes the benefits of the sparse representations of LR and HR patches. The HR image prediction stage providing feed forward neural network model. The second technique is based on up-sampling for IR images using SKLI. The third scheme is depended on up-sampling for IR images using TKLI. This approach is used to increase the sampling rate of a image. The obtained results prove that the success of three presented techniques in acquisition HR of IR images. By comparing between all techniques, it is clear that the second technique gives superior than REI and LSI schemes from point views PSNR and computation time. On the other hand the third technique is the best models from point views PSNR and computation time to other techniques.. The obtained results have shown good visual quality and a promising scheme to apply more SR algorithms on IR images.