Pre-processed Hyperspectral Image Analysis Using Tensor Decomposition Techniques

Abstract

Hyperspectral remote sensing image analysis has always been a challenging task and hence there are several techniques employed for exploring the images. Recent approaches include visualizing hyperspectral images as third order tensors and processing using various tensor decomposition methods. This paper focuses on behavioural analysis of hyperspectral images processed with various decompositions. The experiments includes processing raw hyperspectral image and pre-processed hyperspectral image with tensor decomposition methods such as, Multilinear Singular Value Decomposition and Low Multilinear Rank Approximation technique. The results are projected based on relative reconstruction error, classification and pixel reflectance spectrums. The analysis provides correlated experimental results, which emphasizes the need of pre-processing for hyperspectral images and the trend followed by the tensor decomposition methods.

1 Introduction

Hyperspectral images (HSI) are acquired using air-borne or space-borne hyperspectral sensors [13] such as NASA’s AVIRIS (Airborne Visible/Infrared Imaging Spectrometer) and Hyperion Imaging Spectrometer. Hyperspectral sensors are capable of capturing a pixel at hundreds of contiguous wavelengths and forms a large image cube of reflectance values. The study has revealed that the hyperspectral data is spatial and spectral correlated [13] and the need of spatio-spectral processing for HSI [8]. There are various feasible approaches in literature for spatio-spectral HSI processing and one among them is processing HSI as a third order tensor.

Spectral-spatial dimensionality reduction and HSI classification is based on tensor modelling is proposed by Bourennane et al. where he used Adaptive Multidimensional Weiner filtering (AMWF), a denoising multidimensional filter [3]. Zhang et al. proposed Higher Order Singular Value Decomposition (HOSVD) as an effective compression algorithm for hyperspectral images [13]. Zhang used Khatri-Rao product for reconstruction of the decomposed tensor to reconstruct them back to the original size. [12] projected Additive Morphological Decomposition (AMD) algorithm for HSI and the comparison of classical Principal Component Analysis (PCA) with Tensor-Principal Component Analysis (TCA) based on HSI classification.

Lower Rank Tensor Approximation (LRTA) has been used by Renard et al. [9] for efficient denoising of spatial data and simultaneously reducing the spectral dimension with low rank approximations. Rank-1 tensor decomposition technique is employed by Guo et al. as a denoising tool for HSI. Most of the tensor decomposition algorithms, encompasses that these algorithms can be used as denoising and dimensionality reduction/compression of HSI.

In this paper, HSI is modelled as a third order tensor and behaviour of tensor decomposition methods such as, Multilinear Singular Value Decomposition (MLSVD) and Low Multilinear Rank Approximation (LMLRA) on HSI is analyzed. The tensor decompositions is applied for raw hyperspectral data (without any pre-processing), data normalized hyperspectral data and Least Square Fast Fourier Transform (LS-FFT), a spectral denoising algorithm proposed in a prior work by Chippy et al. [4]. The experimental analysis is projected based on relative reconstruction error obtained after decomposition and reconstruction of HSI, Support Vector Machine (SVM) classification results and pixel reflectance spectrums.

The following section details about the pre-processing techniques employed and the tensor decomposition methods used. The experimental flow of techniques is explained in the Sect. 3. Section 3 also exhibits the experimental results and the analysis derived from the experimental results obtained. The section includes dataset briefing, fixing of compression size for tensor decompositions based on relative reconstruction error and the results obtained from SVM classification and the pixel reflectance spectrums for hyperspectral data with and without pre-processing. Section 4 concludes with the summary of analysis derived from the experimental results and discusses about the future work possible in the discussed research area.

2 HSI Pre-processing Techniques and Tensor Decomposition Methods

2.1 HSI Pre-processing Techniques

The pre-processing of HSI is always an active research area. Data normalization and LS-FFT denoising technique are chosen as two pre-processing techniques for the scope of this paper.

Data Normalization. HSI data is normalized with respect to spectral dimension. Data is normalized to the range of minimum reflectance value and maximum reflectance value captured for that pixel. A vector “x” can be normalized to its min-max range using,

$$\begin{aligned} x'_i = ((x_i- min(x))/(max(x)-min(x))) * ((max(x)-min(x)) + min(x)) \end{aligned}$$

(1)

where, $x'_i$ is the normalized vector of x and i ranges to the length of the vector.

LS-FFT Denoising. Least Square based denoising technique for one-dimensional signals [10] is proposed by Ivan W Selesnick as,

$$\begin{aligned} {\min _\mathbf x }||\mathbf y - \mathbf x ||_2^2 + \lambda ||D \mathbf x ||_2^2 \end{aligned}$$

(2)

where ‘$\mathbf y$’ is the input noisy signal, ‘$\mathbf x$’ is the unknown denoised signal and ‘D’ is the second derivative coefficient matrix; $\lambda$ is the control parameter, which balances the importance of removing noise with retaining the information in the Eq. (2). The solution of the optimization problem mentioned in the Eq. (2) is derived as,

$$\begin{aligned} \mathbf x = {(I + \lambda {D^T}D)^{ - 1}}{} \mathbf y \end{aligned}$$

(3)

Chippy et al. introduced a variant approach to the Ivan W Selesnick’s optimization problem (Eq. 2) to avoid the complex inverse calculation as in the Eq. 3. Chippy adopted denoising in the frequency domain [4] by rewriting the Eq. 2 as,

$$\begin{aligned} {\min _\mathbf x }||\mathbf y - \mathbf x ||_2^2 + \lambda ||\mathbf s \otimes \mathbf x ||_2^2 \end{aligned}$$

(4)

where, $\mathbf s \otimes \mathbf x$ represents the convolution of $\mathbf s$ and $\mathbf x$. Solving the convolution in frequency domain becomes multiplication.

For LS-FFT of matrix, the optimization problem can be formulated as,

$$\begin{aligned} {\min _\mathbf X }||\mathbf Y - \mathbf X ||_2^2 + \lambda ||\mathbf SX ||_2^2 \end{aligned}$$

(5)

where, the discrete Fourier transform of $\mathbf y$, $\mathbf x$, and $\mathbf s$ are represented as ‘$\mathbf Y$’, ‘$\mathbf X$’ and ‘$\mathbf S$’ respectively. The solution is the Inverse Discrete Fourier Transform of the Eq. 5 is,

$$\begin{aligned} \mathrm{{X_k = Y_k/(1 + }}\lambda |S_k{|^2}\mathrm{{)}} \end{aligned}$$

(6)

where k varies from 1 to length of the signal.

2.2 Tensor Decomposition Methods

Multilinear Singular Value Decomposition (MLSVD). Multilinear Singular Value decomposition is a speculation of Singular Value Decomposition for higher dimensional datasets [5]. A third order tensor of size ($I_1 \times I_2 \times I_3$) is decomposed into a set of core tensor and three orthogonal factor matrices. A pixel at i1, i2, i3 location of the tensor ‘a’ can be represented as [5],

$$\begin{aligned} a_{i_{1} i_{2}i_{3}} = \sum _{j_1 = 1}^{I_1} \sum _{j_2 = 1}^{I_2} \sum _{j_3 = 1}^{I_3} s_{j_{1} j_{2}j_{3}} u_{i_{1} j_{1}}^{(1)} u_{i_{2} j_{2}}^{(2)} u_{i_{3} j_{3}}^{(3)} \end{aligned}$$

(7)

where, $u^{(1)}, u^{(2)}, u^{(3)}$ are the orthogonal factor matrices along the three dimensions/modes and s is the core tensor. For a tensor ‘T’, MLSVD can be defined as shown in Fig. 1

Fig. 1.

(Image Courtesy: [2])

Multilinear Singular Value Decomposition

and can be written as,

$$\begin{aligned} T = S ._1 U^{(1)} ._2 U^{(2)} ._3 U^{(3)} \end{aligned}$$

(8)

where S is the core tensor and $U^{(1)}$, $U^{(2)}$, $U^{(3)}$ are the orthonormal bases for three different subspaces of third order tensor. The $._1, ._2, ._3$ or $\times _1, \times _2, \times _3$ representation mentions the first order, second order and third order (n-order) tensor products respectively [7].

Low Multilinear Rank Approximation (LMLRA). Low Multilinear Rank Approximation is comparable with Multilinear Singular Value Decomposition and is different with the way of computation and the optimality of the techniques [1]. LMLRA is computed in two stages, processing the underlying theory and refining the registered yield from the initial step utilizing the mentioned algorithm. The approximation technique tries to limit the frobenius error in each iteration. [6] The low rank approximation techniques Higher Order Orthogonal I and trust-region-based algorithm is discussed by Ishteva and reasons that the trust-region-based algorithm focalizes to the arrangement speeder.

LMLRA Reconstruction Technique. The decomposed tensor can be reconstructed by using Low Multilinear Rank Approximation (LMLRA) reconstruction/regeneration technique. The technique follows a simple Khatri-Rao product (n-order tensor product) of core tensor and factor matrices with respect to its dimensions. The HSI tensor is reconstructed to its original size for comparative analysis.

2.3 Analysis Methods

• Relative Reconstruction Error: Relative reconstruction error is the frobenius norm of the difference in the reconstructed image from the original image to the frobenius norm of the original image [2] and is computed as,

  $$\begin{aligned} RRE = \frac{{||Original\ Image\ -\ Reconstructed\ Image||}_F}{||Original\ Image||_F} \end{aligned}$$

  (9)

  where RRE denotes Relative Reconstruction Error; $||x||_F$ denotes the frobenius norm of x.

• SVM Classification: Hyperspectral data classification follows pixel-wise classification. Each pixel is mapped to a corresponding label. SVM is a linear binary classifier, that classifies two classes with respect to minimize the error function. SVM can be extended for multi-class classification as one-on-one or one-on-all methods.

• Pixel Reflectance Spectrum: Pixel reflectance spectrums are graphical plots of reflectance value plotted against the wavelength/band number of a pixel, which is the spectral signature of the pixel from a remote sensing perspective.

3 Experimental Procedure, Results and Observations

The raw hyperspectral image is compressed using MLSVD and reconstructed to its original size, whose relative reconstruction error (computed using Eq. 9) is noted. The reconstructed image is then analyzed based on the variation in SVM classification results, overall classification accuracy, class-wise accuracy and classification maps, and Pixel Reflectance Spectrums for a pixel chosen from the noisy part of the image. This flow of experiments is repeated for LMLRA compression also, and results are tabulated. As shown in the Fig. 2, the whole set of experiments for both MLSVD and LMLRA is repeated for normalized hyperspectral data and LS-FFT denoised hyperspectral data.

Fig. 2.

Experimental flow - block diagram

3.1 Dataset/Hyperspectral Data

Experiments are carried out on the standard dataset of hyperspectral images: Indian Pines, Salinas Scene and Pavia University [4, 11]. Indian Pines is captured by NASA AVIRIS sensor. Indian Pines was captured over the Indian Pines in North-Western Indiana with $145\times 145$ pixels and 224 bands in the range of 400–2500 nm. Salinas Scene is also acquired by NASA AVIRIS sensor. Salinas Scene was captured over Salinas valley, California with a spatial resolution of 3.7 m pixels. The area covered consists of 512 scan-lines by 217 samples over 224 spectral bands. Pavia University was acquired by the ROSIS sensor over the region Pavia in Northern Italy consisting of $610\times 340$ pixels (excluding the blacked out pixels) and 103 bands.

Fig. 3.

Graphical plot for tuning of compression size for Indian Pines dataset

Table 1. Tuning results and corresponding compression rate

Table 2. SVM classification results

3.2 Tuning of Compression Size Based on Relative Reconstruction Error

The compression size tabulated in the Table 1, is fixed based on the graphical plot plotted for relative reconstruction error against the compression size ranging till the minimum size of the image in all dimensions. A graphical plot result is depicted in the Fig. 3 for the dataset Indian Pines, where the relative reconstruction error tends to become constant at the dimension 45 and hence the compression size is chosen as $45 \times 45 \times 45$ for the experiments performed.

The corresponding relative reconstruction errors for the dataset and tuned compression size and the rate of compression are also recorded in the Table 1. The compression rate is computed as,

$$\begin{aligned} Compression\ Rate = (1 - \frac{No. \ of\ pixels \ of \ Compressed \ Image}{No.\ of\ pixels \ of\ Original\ Image}) \times 100 \end{aligned}$$

(10)

From the Table 1, the relative reconstruction error for data without pre-processing and normalized data are more or less same while there is a huge variation in reconstruction error in case of LS-FFT denoised data.

3.3 SVM Classification Results

Classification of raw data gives dissatisfying results (refer Table 2), hence pre-processing the data becomes vital. As per the overall classification accuracy in the Table 2, it can be noted that relative reconstruction error and Overall Classification Accuracy are inversely proportional which is the expected behaviour. The results also confirms with LS-FFT denoising as an efficient denoising technique and the compression algorithms as lossless. Since the decomposition and reconstruction has only aided in the removal of noise by adding smoothness and has not deprecated the information contained within the image.

Fig. 4.

Sample classification map (Dataset: Indian Pines): (i) Ground truth; (ii) Original image; (iii) Class labels; (iv) MLSVD without pre-processing; (v) MLSVD after data normalization; (vi) MLSVD after LS-FFT denoising; (vii) LMLRA without pre-processing; (viii) LMLRA after data normalization; (ix) LMLRA after LS-FFT denoising

Sample classification map shown in the Fig. 4 confirms the above derived observations. It can be noted that MLSVD and LMLRA approaches the image in two different manner, so that predictions in one class for MLSVD is better than LMLRA and vice-versa. The same can be noted in all the three datasets. The choice of decomposition algorithm wholly depends on the data and should be experimentally chosen. From the overall classification accuracy and classification maps, MLSVD provides better results for both Indian Pines and Pavia University, while LMLRA provides better results for Salinas Scene.

For deep analysis, class-wise accuracy is also considered and tabulated for all the three datasets Indian Pines, Salinas Scene and Pavia University in the Tables 3, 4 and 5. Except for very few classes, the accuracy has increased prominently for most of the classes, assisting the results derived from the classification maps and overall classification accuracy.

Fig. 5.

Pixel reflectance spectrum - Indian Pines (Left: MLSVD, Right: LMLRA)

Fig. 6.

Pixel reflectance spectrum - Salinas Scene (Left: MLSVD, Right: LMLRA)

Fig. 7.

Pixel reflectance spectrum - Pavia University (Left: MLSVD, Right: LMLRA)

3.4 Pixel Reflectance Spectrum

Pixel Reflectance Spectrum is the spectral signature of a pixel. Reflectance spectrums helps us to understand the pixel level information and variation along the wavelength spectrum. Figures 5, 6 and 7 represents the reflectance spectrum of a pixel from the dataset Indian Pines, Salinas Scene and Pavia University. It can be observed that the data without pre-processing and data normalized MLSVD and LMLRA are similar and not differential but varies from the original pixel information denoting the smoothening effect of decomposition algorithm. Spectrum with LS-FFT denoised is prominent and vary from the other reflectance spectrums by the smoothening it provides at highly variation reflection values. This extra smoothening is the denoising effect and reasons the improvement in the classification results and the relative reconstruction error.

Table 3. Class-wise accuracy for various processing - Indian Pines

Table 4. Class-wise accuracy for various processing - Salinas Scene

Table 5. Class-wise accuracy for various processing - Pavia University

4 Conclusion

The paper aims on the comparative study of behaviour analysis of tensor decomposition methods MLSVD and LMLRA with the same reconstruction technique on the raw, data normalized and LS-FFT denoised hyperspectral data. From the experimental results and observations, it can be concluded that the choice of tensor decomposition based on efficiency depends on the data. In terms of computational time, MLSVD is 98% faster than LMLRA. The analysis of other tensor decomposition methods and including other analyzing methods or extending the techniques to other three dimensional data can be considered as the future scope of this paper.