Keywords

36.1 Introduction

In remote sensing imaging, image quality is determined by various distortion factors. Of these factors, blur and noise are the most commonly used physical characteristics. As is well known, they are described by the modulation transfer function (MTF) and noise power spectrum (NPS), respectively. It is greatly affects the subsequent image processing and application. Remote sensing image quality analysis not only can be used for guiding on-orbit remote sensing imaging control, but also to make a preliminary assessment of the quality of image, so it has widespread application.

Remote sensing image quality assessment (IQA) can be divided into two methods: subjective evaluation and objective evaluation. Subjective method requires large amount of people in the completely same condition to mark the image and the mean opinion score is used as the final score of the image, which makes it really time-consuming, cumbersome and expensive to conduct for mass remote sensing image data processing. Objective IQA measures aims to predict perceived image quality by human subjects, which are the ultimate receivers in most image processing applications. Depending on the availability of a pristine reference image, which is presumed to have perfect quality, IQA measures may be classified into full-reference (FR), reduced-reference (RR), and no-reference (NR) methods. In the actual application, remote sensing image usually can’t get reference image, so NR IQA has great application advantages.

Since the 1970s, US has developed NIIRS standard [1] and GIQE equation [2], which can give image quality evaluation if remote sensor parameters are known or can be obtained. Remote sensing image is ultimately for human visual perception. A large number of studies show that considering the characteristic of the human visual system (HVS) in IQA is better than those that do not consider the HVS. But It is very difficult to make objective evaluation results match human visual perception. In recent years some HVS based evaluation models [3, 4] were proposed, but these evaluation models are mainly for a particular type of image distortion [5–8]. An imaging system may only be superior in one metric while being inferior in other metrics.

In this paper, based on the analysis of the HVS and in-depth understanding of the influence of noise and blur on remote sensing image quality, we proposed a NR remote sensing IQA method based on visual information fidelity (VIF) index called PVIF. Experimental results show that PVIF can well reflect the visual perception of the image quality effect.

The paper is organized as follows. Human visual characteristics are discussed in Sect. 36.2. Section 36.3 presents the proposed NR IQA method. The experiments are analyzed in Sect. 36.4 and conclusions are drawn in Sect. 36.5.

36.2 Visual Characteristics of Remote Sensing Image

When we use eyes to observe a remote sensing image, the incentive from image is the combination of signal stimulus with different frequencies and amplitude. The human eye’s response to an excitation signal may also be influenced by other incentives. Contrast masking [9] refers to the reduction in visibility of one signal stimulus caused by the presence of another signal stimulus. Due to the existence of visual contrast masking, some distortions of remote sensing image may be ignored by human eyes. Those distortions will not affect the overall image quality; but another distortion will be strengthened, that seriously deteriorate image quality.

Image blur effect is caused by the loss of the high frequency content. It can be reproduced with a low-pass filter. We observe that it is difficult to perceive differences between a blurred image and the same re-blurred image. If we blur a sharp picture, image quality will change with a major variation. On the contrary, if we blur an already blurred picture, image quality will still change, but only to a weaker extent. In Fig. 36.1, we present from left to right the original sharp image, the original image blurred with a low-pass filter and the blurred image re-blurred with the same low-pass filter. We observe a high difference in term of loss of details between the first and the second image and a slight difference between the second and the third image. We can explain this phenomenon by the fact that the second blurring effect reduces the difference between pixels that has already been reduced by the first blurring effect. If we add noise to an already noised image, due to the existence of visual contrast masking of HVS, we notice that the high differences significantly decrease after the first noising step and slightly decrease after the second noising step as shown in Fig. 36.2.

Fig. 36.1
figure 1

From left to right: original image, original image blurred with an averaging filter, blurred image re-blurred with the same averaging filter

Full size image
Fig. 36.2
figure 2

From left to right: original image, add white Gaussian noise to original image, noised image re-noised by add the same white Gaussian noise

Full size image

36.3 The Proposed Method

The key idea of our IQA principle is to re-blur and re-noise the input image and to analyze the behavior of the mutual information. As one of the most popular FR IQA method, the Visual Information Fidelity (VIF) [10] takes an information theoretic framework of visual contents based on natural scene statistics models. We first give a brief introduction of the VIF index, and then propose our method.

36.3.1 VIF index

Figure 36.3 provides an overview of the VIF. Let \( C \) pass through HVS, which is modeled as a loss channel, and call the output \( E \). Let a distorted image \( D \) also be subjected to HVS loss, and we refer to the output of HVS channel as \( F \). In the view of information theory, mutual information between \( C \) and \( E \) (\( I(C,E) \)) reveals the amount of information that “loss channel” preserves about the input \( C \). VIF interprets this mutual information as a way to assess image quality in HVS.

Fig. 36.3
figure 3

VIF flow diagram

Full size image

Let \( C \) and \( D \) denote the random fields (RFs) from the reference and distorted images respectively. \( C \) is a product of two stationary RFs that are independent of each other: \( C = SU = \{ S_{k} U_{k} :k \in I\} , \) Where \( I \) denotes the set of spatial indices for the RFs, \( S \) is a RFs of positive scalars, and \( U \) is a Gaussian scalar RFs with mean zero and variance \( \sigma_{U}^{2} \).

The image distortion model is a signal attenuation and additive Gaussian noise, defined as \( D = GC + V = \{ g_{k} C_{k} + V_{k} :k \in I\} , \) where \( G \) is a deterministic scalar attenuation field, and \( V \) is a stationary additive zero-mean Gaussian noise RFs with variance \( \sigma_{V}^{2} \).

The human visual system (HVS) model in VIF quantifies the impact of the image that flows through HVS: \( E = C + N \) and \( F = D + N, \) where \( E \) and \( F \) denote the cognitive outputs of the reference and test images extracted from the brain, respectively; \( N \) represents stationary white Gaussian noise RFs with variance \( \sigma_{n}^{2} \).

VIF utilizes mutual information \( I(C_{k} ,E_{k} ) \) to measure the information that can be extracted from the output of HVS when the reference image is being viewed

$$ I(C_{k} ,E_{k} ) = \frac{1}{2}\log_{2} \left( {\frac{{\left| {s_{k}^{2} C_{U} + \sigma_{N}^{2} I} \right|}}{{\left| {\sigma_{N}^{2} I} \right|}}} \right) = \frac{1}{2}\log_{2} \left( {1 + \frac{{\sigma_{{C_{k} }}^{2} }}{{\sigma_{N}^{2} }}} \right). $$
(36.1)

In addition, information \( I(C_{k} ,F_{k} ) \) is measured in the same way when the test image is being viewed

$$ I(C_{k} ,F_{k} ) = \frac{1}{2}\log_{2} \left( {\frac{{\left| {g_{k}^{2} s_{k}^{2} C_{U} + (\sigma_{N}^{2} + \sigma_{{V_{k} }}^{2} )I} \right|}}{{\left| {(\sigma_{N}^{2} + \sigma_{{V_{k} }}^{2} )I} \right|}}} \right) = \frac{1}{2}\log_{2} \left( {1 + \frac{{g_{k}^{2} \sigma_{{C_{k} }}^{2} }}{{\sigma_{N}^{2} + \sigma_{{V_{k} }}^{2} }}} \right) $$
(36.2)

Also, we have only dealt with one sub-band so far. One could easily incorporate multiple sub-bands by assuming that each sub-band is completely independent of others in terms of the RFs. The VIF index assesses mutual information between \( C \) and \( E \) (and \( C \) and \( F \)) as follows:

$$ {\text{VIF}}(C,D) = \frac{{\sum\limits_{j \in subbands} {\sum\nolimits_{k} {I(C_{k}^{j} ,F_{k}^{j} )} } }}{{\sum\limits_{j \in subbands} {\sum\nolimits_{k} {I(C_{k}^{j} ,E_{k}^{j} )} } }} = \frac{{\sum\limits_{j} {\sum\limits_{k}^{{}} {\log_{2} \left( {1 + \frac{{\left( {g_{k}^{j} } \right)^{2} \sigma_{{C_{k}^{j} }}^{2} }}{{\sigma_{N}^{2} + \sigma_{V}^{2} }}} \right)} } }}{{\sum\limits_{j} {\sum\limits_{k}^{{}} {\log_{2} \left( {1 + \frac{{\sigma_{{C_{k}^{j} }}^{2} }}{{\sigma_{N}^{2} }}} \right)} } }} $$
(36.3)

36.3.2 The Proposed NR IQA Based on VIF Index

In order to measure the quality of remote sensing images, we first obtain a re-blurred image by filter the input image with Gaussian kernel low-pass filter, then add white Gaussian noise to input image and get a re-noised image. As we discussed in the previous section, we measure the amount of information changes before and after image filtering and noising. We take the VIF index value as a measure of this information changes. Bigger VIF values represent smaller image information changes. We obtained the final IQA results by combining information changes at each pixel. Fig. 36.4 shows a flowchart of the proposed NR IQA algorithm, and the whole steps are as follows.

Fig. 36.4
figure 4

Flowchart of proposed algorithm

Full size image

Step 1: A re-blurred image \( I_{b} \) is produced by applying Gaussian filter to the input image \( I_{0} \).

Step 2: A re-noised image \( I_{n} \) is produced by adding white Gaussian noise to the input image \( I_{0} \).

Step 3: Compute \( {\text{VIF}}(I_{0} ,I_{b} ) \) and \( {\text{VIF}}(I_{0} ,I_{n} ) \) using Eq. (36.3).

Step 4: The \( {\text{VIF}}(I_{0} ,I_{b} ) \) and \( {\text{VIF}}(I_{0} ,I_{n} ) \) are used to construct the final IQA index by compute\( {\text{PVIF(}}I_{0} ) { = }\left( {1 - {\text{VIF}}(I_{0} ,I_{b} )} \right)\left( {1 - {\text{VIF}}(I_{0} ,I_{n} )} \right) \).

36.4 Experimental Results

In this section, we test the proposed NR IQA method on some remote sensing images, compared with FR-IQA method VIF index and PSNR. We use 4 typical remote sensing images from worldview-2 satellite as test images shown in Fig. 36.5. Then we use these 4 test images to generate a series of distorted images denoted as C1, C2, C3 and C4 respectively by adding different Gaussian blur and Gaussian noise. Spearman rank-order correlation coefficient (SROCC) [11] is used to assess performance of the quality index.

Fig. 36.5
figure 5

Test images used in the experiments

Full size image

Table 36.1 lists the SROCC results of PVIF and the two IQA algorithms on the C1, C2, C3 and C4. From Table 36.1, we can see that the proposed VIF based IQA metric PVIF performs consistently well across all the test images. The SROCC between PVIF and VIF are greater than 0.92 in all 4 test images, and exceed 0.6 when compared with PSNR. This means when the reference image does not exist, PVIF can replace VIF and PSNR and give a more accurate quality evaluation. Figure 36.6 shows the scatter distributions of VIF versus the predicted scores by PVIF. The curves shown in Fig. 36.6 were obtained by a nonlinear fitting [12]. From Fig. 36.6, one can see that the objective scores predicted by PVIF correlate consistently with the VIF index.

Table 36.1 The SROCC performance of PVIF compare with VIF and PSNR
Full size table
Fig. 36.6
figure 6

Plots of VIF versus PVIF of the image sets degraded from 4 test images

Full size image

36.5 Conclusion

We have presented a novel robust, low-cost no-reference remote sensing image quality assessment algorithm. The mutual information between an original image and its re-blurred and re-noised versions has been proposed to estimate image quality. The proposed method has been shown to have robust estimation.