Keywords

Introduction

With rapid advances in digital technology and communication, cybercrime activities have dramatically increased. Surveillance cameras and mobile device embedded cameras have been a major source of information for forensic/criminal investigations. Digital image/video footage of the crime or cybercrime activity can be used to reconstruct the sequence of events, identify the criminal/s, confirm the time and location of the crime and other types of evidence in the legal proceedings. Hence, the quality of the recorded image/video footage, is crucial for any forensic/criminal investigation and subsequently legal proceedings. However, the quality of the recoded image/video footage is sometimes insufficient for the investigation purpose due to the distance and angle of the camera from the scene. This causes the object of interest for example the face of a person in the scene to be of low resolution which increases the difficulty of the recognition process. Therefore, the application of the image/video resolution enhancement technique to improve the quality of the footages is necessary.

Traditional image interpolation methods such as Bilinear, Bicubic, Bspline and nearest neighbourhood, do not provide sufficient visual quality particularly around sharp edges due to use of local smoothness filters [1]. Wavelet based super-resolution techniques in some extents could mitigate the over smoothing problem of the traditional image resolution enhancement techniques by using a set of filter banks. A wavelet-based interpolation method was proposed by Carey et al. in [1]. The proposed method estimates the wavelet high frequency coefficients of the input image by exploiting the regularity of the edges across the scales. Authors reported significant improvements over the traditional interpolation methods. However, this method may not be able to estimate the wavelet high-frequency coefficients with small values, which could reduce the effectiveness of this technique.

Demirel and Anbarjafari [2] reported a Complex Wavelet Transform (CWT) based image resolution enhancement technique. The proposed algorithm applies a dual-tree CWT (DT-CWT) to the input image, decomposing the input image into its frequency subbands. The resolution enhancement is achieved by using directional selectivity provided by the CWT, where the high-frequency subbands in six different directions contribute to the sharpness of the high-frequency details. An inverse DT-CWT is then applied to the coefficients in the high-frequency subbands to generate the high-resolution image. A Dual-Tree Complex Wavelet Transform (DT-CWT) and a Non-Local Means (NLM) based image resolution enhanced technique were reported in [3]. In this technique, the high frequency subbands are first generated using a DT-CWT. Windowed form of the Sinc filter are then employed to interpolate the high-frequency subbands and the low resolution input image. The NLM filter is then applied to the high frequency subbands to reduce artefacts generated by the DT-CWT. Finally, an inverse DT-CWT combines the resulting high frequency subbands and the low resolution input image, generating the super-resolution image. They reported significant improvements over the state of the art techniques, both subjectively and objectively.

A wavelet based image resolution enhancement technique was proposed in [4]. The proposed algorithm uses a Discrete Wavelet Transform (DWT) to decompose the input image into its frequency subbands. The resulting frequency subbands are then interpolated using the Bicubic algorithm. The difference between the input low resolution image and the interpolated low frequency subband is then used to refine the interpolated high frequency subbands. An inverse DWT is finally used to generate the super-resolution image. Authors reported superior results compared to the traditional and state-of-art image resolution enhancement techniques. Temizel and Vlachos proposed another wavelet based image resolution enhancement technique using Cycle Spinning (CS) [5], which outperformed other state of the art techniques, at the time. The authors of this paper reported another wavelet based image resolution enhancement algorithm, which operates in a quad-tree wavelet decomposition framework and exploits wavelet coefficient correlation in a local neighbourhood sense [6]. This method employed linear least-squares regression algorithm to estimate the wavelet high-frequency coefficients and generates superior results compared to the conventional methods for a wide range of test images.

However, the application of wavelet filters in estimating super resolution subbands, which have the potential in improving the quality of the enlarged image, have not been reported in literature. In this paper, wavelet filters are used to estimate coefficients in high frequency subbands of the super resolution image, from the detail subbands of the input low resolution image. Results demonstrate the merit of the proposed technique. The rest of the paper is organized as follows: the proposed technique will be explained in Sect. 2, experimental results are given in Sect. 3 and finally paper will be concluded in Sect. 4.

Wavelet Based Image Enlargement Technique

Figure 1 shows a block diagram of the proposed Wavelet based Image Enlargement (WIE) technique. A low resolution image is input to the system.

Fig. 1.
figure 1

Block diagram of the proposed image enlargement technique.

Full size image

A two dimensional wavelet transform is applied on the input image, transforming it into its frequency subbands called: baseband (LLLR), and LHLR, HLLR and HHLR high-frequency subbands. Estimating edge information is essential in generating a high quality super resolution image. In the proposed WIE algorithm, information in high frequency subbands of the input image are used to estimate the edge information, within the high frequency subbands’ of the super resolution image. To achieve this, the high frequency subbands of the low resolution image are processed as follows:

Each column of the LHLR subband is first up-sampled by a factor of two, to double its height. Each resulting column of the LHLR is then filtered using the wavelet transform high-pass reconstruction filter, named hpfr. Each row of the resulting filtered up-sampled LHLR subband is then up-sampled by a factor of two, doubling the width of the LHLR subband. Each resulting row is then filtered using the wavelet transform low-pass reconstruction filter, called lpfr, generating an estimation of the coefficients in the LH subband of the super-resolution image. The other two low-resolution subbands, HLLR and HHLR, are processed using a similar method. However, the lpfr and hpfr filters are used for filtering information in each column of the HLLR and HHLR subbands, respectively and the hpfr filter is used for filtering coefficients in each row of the HLLR and HHLR subbands. The above processes generate estimations for coefficients in high frequency subbands’ of the super resolution image, called: LHLR, HLLR and HHLR.

The resulting subbands and input low resolution image, which is assumed to be an estimation of the super resolution baseband image called LL, are then used to create the super resolution image by applying a two dimensional inverse wavelet transform on the subbands.

Experimental Results

To generate experimental results, five standard test images which contain a wide range of frequency components, called: Lena, Elaine, Peppers, Mandrill, and Barbara were taken. Each of these five images, were first lowpass filtered using a 2D Blackman lowpass filter with a cutoff frequency of Fs/2 to mitigate the aliasing artefact of the down sampling. The filtered images were then down sampled by a factor of 2 in both horizontal and vertical directions, generating a replica low resolution image for each of the input images. Hence, the input images could be treated as the ground truth for comparing the performance of the image enlargement techniques. The Blackman 2D FIR filter coefficients are tabulated in Table 1 [7].

Table 1. The Blackman 2D FIR filter coefficients [7].
Full size table

To evaluate the performance of the proposed Wavelet based Image Enlargement (WIE) technique, the resulting low resolution Lena, Elaine, Peppers, Mandrill, and Barbara test images were enlarged using the proposed WIE method, Nearest-neighbourhood, Bilinear, Bicubic, Sinc, Cycle Spinning (CS) [5], Directional Cycle Spinning (DSC) [2] techniques. Bi-orthogonal Daubechies 9/7 wavelet filters were used for generating experimental results. These filters were chosen because they are commonly used in image processing and image coding applications; but the proposed WIE algorithm works with most other wavelet and subband filters as well, providing that the filters are of sufficient length to yield basis functions that are more regular than the signals being analyzed.

The Peak Signal to Noise Ratio (PSNR) measurement and the Structural SIMilarity (SSIM) index, which has proven to be inconsistent with human eye perception, were chosen to assess the quality of the enlarged images against their corresponding ground-truth images (the original test images were assumed to be the ground truth images). The PSNR measurements and the SSIM indexes for the enlarged images were calculated and tabulated in Tables 2 and 3, respectively. From Tables 2–3 it can be seen that the proposed WIE technique provides the most improvement over the state of the art methods, both subjectively and objectively. From Table 2, it can be seen that the proposed technique generates a slightly lower PSNR when enlarging Lena and Mandrill images in comparison to the Sinc technique. However, it is well known that the PSNR is not a reliable metric to judge the visual quality of the enlarged images and the human eyes are the final judges for assessing the quality of the images. Hence, the SSIM index, which is more inconsistent with the perception of the human eyes, is a more reliable metric to be used for evaluating the quality of the enlarged images and this index shows that the proposed technique outperforms the state of the art techniques.

Table 2. The PSNR comparison of different methods.
Full size table
Table 3. The SSIM comparison of different methods.
Full size table

To give a visual perception of the resulting enlarged images, a section of the original Barbara test image, its generated low resolution image, and its enlarged image using the Sinc, Cycle Spinning (CS), Directional Cycle Spinning (DSC) and the proposed technique were shown in Fig. 2. From Fig. 2, it can be seen that the enlarged image using the proposed technique is the closest to the original image. It is also clear that the enlarged image using CS and DSC suffer from general blurriness and the Sinc image has less blur with superior contrasts, however it suffers from pixilation within the smooth areas. At the same time, the proposed technique produces an image with high contras and less pixilation in smooth areas while preserving precise edges.

Fig. 2.
figure 2

A section of Barbara image: (a) Ground truth image (b) low resolution image and enlarged image using (c) Sinc, (d) cycle spinning, (e) directional cycle spinning and (f) proposed techniques.

Full size image

Conclusions

In this paper, a new image resolution enhancement technique was proposed. The proposed technique estimated the wavelet coefficients of the high frequency subbands of the super resolution image from the wavelet subbands of the low resolution image by up-sample and then filtering the low resolution detail subbands using inverse wavelet filters. Results showed that the enlarged images using the proposed technique exhibit significantly higher visual quality to that of the state of the arts techniques, subjectively and objectively.