Keywords

1 Introduction

In recent, the image registration techniques have been developed for many applications and data [1]. Remote sensing image registration is an essential step in analysis of remote sensing image. The registration is a critical process for essential image processing and for image fusion [2]. It has many applications such as change detection, image fusion and etc. [3, 4]. Image fusion is combining two images in a single image. The accuracy of image registration has great effect on the final fusion result. If a panchromatic image and a multi-spectral image have been registered precisely. The final fused image will appear as one image, although it integrate two images. Otherwise, the final fusion image will appear blurry or show edge phenomena because of mis-registration [5]. There are many of the image registration techniques. Techniques deal with the intensity values without detecting salient features. These techniques are substantial to the intensity distribution. In [6–8] the feature-based techniques use salient features extracted from two images do not directly work with image intensity values [5, 9–11], which appears more suitable for such situations that intensity changes and complicated geometric deformations are encountered. Therefore, these feature based techniques have been widely used in remote sensing image registration. In literature, there are several remote sensing image registration techniques. There many remote sensing image registration techniques based on multi-scale transform like steerable pyramid transform and scale invariant feature transform [12] and the scale invariant feature transform [13, 14]. This paper proposes an optimization approach for remote sensing image registration. This approach is used for determining pairs of corresponding points between images. The approach implements the particle swarm optimization (PSO). The PSO used as a function optimizer and mutual information (MI) is used as a similarity measure. Firstly, Landmarks were chosen manually and used thin plate spline (TPS) to provide a geometric representation for the relative locations of corresponding landmarks. Secondly, MI used as a cost function to determine the degree of similarity between two images. Finally, PSO is used to maximize MI function and to improve the correspondence between the landmarks. this paper is organized as the following: Sect. 2 gives an insight about thin plate spline and its implementation in the image registration process, the Mutual Information concepts and its important in the image registration process is presented and the novel of the Particle Swarm Optimization is explained and how it is implemented. Section 3 proposes the optimize and mutual information and thin plate spline (OMI_TPS) method to register remote sensing images and presents a mode of its work. Section 4 presents the outline of the test data and the experimental results. Finally, Sect. 5 presents the conclusion that is drawn depend on Sect. 4 the experimental results.

2 Preliminaries

2.1 Thin Plate Spline (TPS)

TPS or surface splines are perhaps the most widely used transformation functions in the registration of images with nonlinear geometric differences. It was first used by Goshtasby [15] in the registration of remote sensing images and the registration of medical images [16, 17]. TPS is the process of computing the transformation parameters then finding interpolation function, which minimizes the bending energy, following is a brief algebraic description of the TPS model [18]. If \( p = x_{i} ,y_{i} \)) and \( q = x_{i} ,y_{i} \)), \( i = j = 1, \ldots ,n \) represent two sets of corresponding LMs in a reference and target images respectively, then the TPS interpolant \( f(x,y) \) minimizes the bending energy.

$$ E(z(x,y)) = \int \int \left[ {(\frac{{\partial^{2} z(x,y)}}{{\partial x^{2} }})^{2} + (\frac{{\partial^{2} z(x,y)}}{\partial xy})^{2} + (\frac{{\partial^{2} z(x,y)}}{{\partial y^{2} }})^{2} } \right]{\text{d}}x{\text{d}}y $$
(1)

And has the form

$$ f(x,y) = a_{1} + a_{x} x + a_{y} y + \sum (W*U(\left| {(x_{i} ,y_{i} ) - (x,y)} \right|)) $$
(2)

Vector a defines a fine part of the mapping and vector denotes the nonlinear part of the deformation, that were determined by

$$ \frac{W}{{a_{1} a_{x} a_{y} }} = inv(L)*q $$
(3)

where

$$ L = \left[ {\begin{array}{*{20}c} k & p \\ {p^{T} } & 0 \\ \end{array} } \right] $$
(4)

where \( K \) is a \( n \times n \) matrix and \( K_{i} j =U(\left\| {(x_{i} ,y_{i} ) - (x_{j} ,y_{j} )} \right\|) \), where \( U(r) = r^{2} { \log }(r) \), \( i{\text{th}} \) row of \( p \) is \( (1,x_{i} ,y_{i} ) \), 0 is a 3 × 3 matrix of zeros. The TPS framework is used to warp the moving image using nearest neighbor interpolation to map the pixel values onto integer coordinates. Figure 1 given in [19] shows the original coordinate of the control points as indicated by the black dot and the surrounding TPS data points arranged as a mesh. Figure 2 given in [19] shows that the data points that is originally arranged in a grid pattern is remapped when the control points as indicated by the black dot undergoes coordinate transformation.

2.2 Mutual Information

Mutual Information (MI) is an important concept of information theory, which measures the statistical dependence between two variables. It is used as a cost function in similarity measure based approaches to registration was first suggested by Viola [20]. It has been extensively exploited for the task of matching correspondence points. It expresses the amount of information that one image contains about a second image. MI considers the information contributed to the overlapping region by each image being registered together with the joint information. MI I(A,B) of the two images and can be defined as following:

$$ I(A,B) = H(A) + H(B) - H(A,B) $$
(5)

where \( H(A) \) and \( H(B) \) denote the separate entropy values of A and B respectively. \( H(A,B) \) means the joint entropy. The joint entropy \( H(A,B) \) measures the amount of information in the combined images, it is given by:

$$ H(A,B) = \sum\limits_{(a \subset A)} \sum\limits_{(b \subset B)} P_{AB} (a,b)\log P_{A} B(a,b) $$
(6)

where (a, b) the joint probability density function of the images and B. Separate entropy values of and B derived from the probabilities of each image:

$$ H(A) = - \sum\limits_{(a \subset A)} P_{A} (a)\log P_{A} (a) $$
(7)
$$ H(B) = - \sum\limits_{(b \subset B)} P_{B} (b)\log P_{B} (b) $$
(8)

where \( P_{A} (a) \) and \( P_{B} (b) \) are the independence probability distributions of image and respectively. Therefore, Eq. (5) can be rewritten as:

$$ I(A,B) = \sum\limits_{(a \subset A)} \sum\limits_{(b \subset B)} P_{AB} (a,b)\log [\frac{{P_{AB} (a,b)}}{{P_{A} (a).P_{B} (b)}}] $$
(9)

The optimal parameters of the spatial transformation T\( \alpha \), which brings the images into registration are found by maximizing MI:

$$ T\alpha = \text{Max}\,I\,(A,B^{t} ) $$
(10)

where I(A, B t) MI of image and is transformed using parameter and Tα is the position at which I(A, B t) is maximized [21].

2.3 Particle Swarm Optimization

PSO is a parallel evolutionary computation technique, it is a global optimization method originally developed by Kennedy and Eberhart in 1995 [22], inspired by social behavior of flocks of birds when they are searching for food. In PSO, each member of the population, also called particle, flies around in the multidimensional search space exploring for better regions. In a flock of birds, the leader exchanges information with the rest of the other birds, while in PSO, the particle leader (the best solution) exchanges information with the rest of the other particles. PSO has already been applied in many areas, such as function optimization, neural network training and pattern classification and fuzzy system control. It is a population-based iterative learning algorithm that shares some common characteristics with other evolutionary computation algorithms. However, PSO searches for an optimum through each particle flying in the search space and adjusting its flying trajectory according to its personal best experience and its neighborhoods best experience rather than through particles undergoing genetic operations like selection, crossover, and mutation. Owing to its simple concept and high efficiency, PSO has become a widely adopted optimization technique and has been successfully applied to many real world problems [23]. The rules are the mechanism that each particle learns from its own best historical experience and its neighborhoods best historical experience. According to the method of choosing the neighborhoods best historical experience, PSO algorithms are traditionally classified into global version PSO (GPSO) and local version PSO (LPSO). In GPSO, a particle uses the best historical experience of the entire swarm as its neighborhoods best historical experience. In LPSO, a particle uses the best historical experience of the particle in its neighborhood which is defined by some topological structure, such as the ring structure, the pyramid structure, or the von Neumann structure [24]. PSO is initialized with a group of random particles and then searches for optima by updating generations. The particles velocity and position update according to the following equations:

$$ V_{id}^{t + 1} = wv_{id}^{t} + c_{1} r_{1} (P_{BESTid}^{t} X_{id}^{t} ) + c_{2} r_{2} (g_{BESTid}^{t} X_{id}^{t} ) $$
(11)
$$ X_{id}^{t + 1} = X_{id}^{t} + V_{id}^{t + 1} $$
(12)

\( P_{BESTid}^{t} \) is the best previous position along the \( d{\text{th}} \) dimension of particle in iteration (memorized by every particle), \( g_{BESTid}^{t} \) is the best previous position among all the particles along the \( d{\text{th}} \) dimension in iteration (memorized in common repository), indicates an inertia weight, \( r_{1} ,r_{2} \in (0,1) \) of random numbers. \( c_{1} ,c_{2} \) is a learning factor, a non-negative constant, set by the user, usually \( c_{1} = c_{2} \). The whole optimization is a repeat a process and stop upon a stooping criterion which is the maximum iteration number or the minimum error condition is met [25].

3 OMI_TPS Method

OMI_TPS method attempts to register images are acquired at from different sensors, the method is proposed for using PSO equations to optimize MI is used as a cost function to determine the degree of similarity between registered images and combined with TPS is used to compute geometric mapping between both images. The proposed method based on employing TPS between the reference and target images for computing the transformation parameters then finding interpolation function, MI is used as similarity metric and PSO is used as optimizer to maximize MI, where MI is maximized when images are correctly registered. The TPS parameters are used to warp target image using interpolation function to map the corresponding points on both reference and target images under the direction of minimizing bending energy, as described in previous section. However, TPS interpolation produces holes in the transformed image as not all the points are being mapped onto it. Therefore, the reverse transformation fails to achieve accurate results since, splines are not exactly reversible. Hence, PSO was proposed to use as a optimizer combined with MI is used as a cost function to fill the holes produced by forward TPS interpolation function. OMI_TPS method attempts to register images by combining TPS with MI, which is very powerful similarity metric in IR where the MI is maximized when images are correctly registered. The method based on employing TPS to compute geometric mapping between the reference and target images through initial LMs, which were selected in both images, then the algorithm automatically refines the position of the LMs under the direction of optimizer by PSO equation to maximize MI of generating optimal correspondences to register images. OMI_TPS method can be summarized as follow, compute geometric mapping through TPS equations then PSO is used to maximize MI to achieve optimal correspondence between images. Details of proposed method is given in algorithm 1, which shows optimizing TPS using PSO, where a stooping criterion defined to be the optimal solution, maximum number of iteration or when optimal solution is not reached and more iterations do not affect the error. The OMI_TPS method attempts to apply an automatic MI based registration algorithm across corresponding points of feature spaces, the algorithm requires minimal user input, which initially set control point pairs for the reference and homologous data sets in the Target, then interpolated mapped data set pair by computing geometric mapping through TPS. The algorithm automatically refines the positions of the control points in the homologous data set under the direction of the PSO equations by moving them to maximize MI cost function. The algorithm automatically refines the positions of the control points in the homologous data set under the direction of the PSO equations by moving them to maximize MI cost function.

4 Experimental Results

In this section, the proposed approach was tested for Satellite images. The effectiveness of the OMI_TPS was evaluated by using different satellites images. Images from Spot and Aster satellite have been used in the experiments. panchromatic image of Spot satellite which is gray image as reference image, Fig. 1a and multispectral image of Aster satellite which is RGB image as target image, Fig. 1b. The two images reference and target image has different spatial resolution. Figure 1c shows the final registered image. The optimization rates of matching error is tabulated in Table 1, which is measured by the number of false matches when establishing the correspondence between LMs candidates, the reducing of errors ratio was tabulated through the iterative optimization process by employing PSO as function optimizer in estimation of MI used as a similarity measure.

Fig. 1
figure 1

Registration Satellite Image Based On Proposed Technique. a Spot panchromatic reference image. b Aster multispectral target image. c Final registered image

Full size image
Table 1 The optimization rates of matching error ratio of OMI_TPS
Full size table

The multi-sensor image fusion is the combining between the low multi-spectral resolution image and the high spatial resolution panchromatic image to produce the high multi-spectral resolution image which is called pan-sharpening. The low multi-spectral and high spatial resolution images must be geometrically registered prior to fusion. The registration image has an impact on the quality of the image fusion. The effective multi-sensor image fusion requires geometric correlation between the two fused images. The wavelet transform image fusion approach is used to show the affect of the proposed the OMI_TPS image registration approach on the quality of image fusion, see Fig. 2. It is clear from the Fig. 2 a lot of fog occurrence of distortion in the image, which prevents their use in various applications. And will use some statistical criteria for measuring image quality such as The correlation coefficient (CC) measures the Convergence between the original image and the result images. the highest correlation coefficient value shows the best correlation between fused and original image data, the entropy information (EI) measures the richness of the information in fused image, the standard deviation (SD) measures the value of the deviation from the mean of the image and the discrete degree between each pixel and the mean value of one image, the smallest value is the best. And the Peak Signal to Noise Ratio (PSNR) measures the quality reconstruction of image. The precise registration of the image achieves the highest quality of image fusion.

Fig. 2
figure 2

Fusion registered image and noncontent…-registered image. a Panchromatic image. b Registered multispectral image. c Fused registered image. d Non-registered multispectral image. e Fused non-registered image

Full size image

Table 2 shows the effect of the image registration on the image fusion quality.

Table 2 The statistical analysis of image fusion quality
Full size table

5 Conclusion and Future Work

Remote sensing Image registration is one of the most important processes when integrating and analyzing information from many different sources. It is an essential stage in image fusion and change detection. The image fusion accuracy depends on the accuracy of the image registration. So, this paper proposes an optimization approach for remote sensing images registration OMI_TPS method. The OMI_TPS method based on the implementation of PSO used as function optimizer combined with transformation parameters is generated by TPS and MI similarity metric as the objective function to be maximized. The algorithm implemented on satellite images. Experimental results demonstrate that OMI_TPS method yields significant improvement in the registration accuracy and it has high performance and accuracy even if the images have different spatial resolution. The experimental results prove that the proposed method has better performance for large scale variations, rotation and intensity changes.