Abstract
Automatic multi-sensor image registration is a challenging task in remote sensing. Conventional image registration algorithms may not be applicable when common underlying visual features are not distinct. In this paper, we propose a novel image registration approach that integrates local self-similarity (LSS) and mutual information (MI) for multi-sensor images with rigid/nonrigid radiometric and geometric distortions. LSS is a well-performing descriptor that captures common, local internal layout features for multi-sensor images, whereas MI focuses on global intensity relationships. First, potential control points are identified by using the Harris algorithm and screened based on the self-similarity of their local surrounding internal layouts. Second, a Bayesian probabilistic model for matching the ensemble of the LSS features is introduced. Third, a particle swarm optimization (PSO) algorithm is adopted to optimize the point and region correspondences for maximum self-similarity and MI and, ultimately, a robust mapping function. The proposed approach is compared with several conventional image registration algorithms that are based on the sum of squared differences (SSD), scale-invariant feature transforms (SIFT), and speeded-up robust features (SURF) through the experimental registration of pairs of Landsat TM, SPOT, and RADARSAT SAR images. The results demonstrate that the proposed approach is efficient and accurate.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Abdel Sayed S, Ionescu D, Goodenough D (1995). Matching and registration method for remote sensing images. In: Proceedings of Geoscience and Remote Sensing Symposium, 2, 1029–1031
Arévalo V, González J (2008). Improving piecewise linear registration of high-resolution satellite images through mesh optimization. IEEE Trans Geosci Remote Sens, 46(11): 3792–3803
Atousa T (2011). Local self-similarity as a dense stereo correspondence measure for thermal-visible video registration. In: Proceedings of the 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,Washington, DC, USA
Bay H, Ess A, Tuytelaars T, Van Gool L (2008). Speeded-up robust features (SURF). Comput Vis Image Underst, 110(3): 346–359
Belongie S, Malik J, Puzicha J (2002). Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell, 24(4): 509–522
Bentoutou Y, Taleb N (2005a). A 3-D space‒time motion detection for an invariant approach image registration approach in digital subtraction angiography. Comput Vis Image Underst, 97(1): 30–50
Bentoutou Y, Taleb N (2005b). Automatic extraction of control points for digital subtraction angiography image enhancement. IEEE Trans Nucl Sci, 52(1): 238–246
Bentoutou Y, Taleb N, Chikr El Mezouar M, Taleb M, Jetto L (2002). An invariant approach for image registration in digital subtraction angiography. Pattern Recognit, 35(12): 2853–2865
Boiman O, Irani M (2007). Detecting irregularities in images and in video. Int J Comput Vis, 74(1): 17–31
Borzi A, Bisceglie M D, Galdi C, Giangregorio G (2009). Robust registration of satellite images with local distortions. In: Proceedings of 2009 IEEE International Geoscience and Remote Sensing Symposium, 3: III-251–III-254
Bouchiha R, Besbes K (2013). Automatic remote-sensing image registration using SURF. International Journal of Computer Theory and Engineering, 5(1): 88–92
Brook A, Ben-Dor E (2011). Automatic registration of airborne and spaceborne image topology map matching with SURF processor algorithm. Remote Sens, 3(1): 65–82
Chen H M, Arora M K, Varshney P K (2003a). Mutual informationbased image registration for remote sensing data. Int J Remote Sens, 24(18): 3701–3706
Chen H M, Varshney P K, Arora M K (2003b). Performance of mutual information similarity measure for registration of multitemporal remote sensing images. IEEE Trans Geosci Remote Sens, 41(11): 2445–2454
Clerc M, Kennedy J (2002). The particle swarm—Explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput, 6(1): 58–73
Cole-Rhodes A A, Eastman R D (2011). Gradient descent approaches to image registration. In: Moigne J L, Netanyahu N S, Eastman R D, eds. Image Registration for Remote Sensing. Cambridge: Cambridge University, 265–276
Cole-Rhodes A, Johnson K L, Moigne J L, Zavorin I (2003). Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Transactions on Image, 12(12): 1495–1511
Cole-Rhodes A, Johnson K, Le Moigne J (2012). Multiresolution registration of remote sensing images using stochastic gradient. In: Szu H H, Buss J R, eds. Wavelet and Independent Component Analysis Applications IX. SPIE Proceedings Vol. 4738
Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, Marchal G (1995). Automated multimodality image registration based on information theory. Inf Process Med Imaging, 3: 263–274
Farah I R, Boulila W, Ettabaâ K S, Solaiman B, Ahmed M B (2008). Interpretation of multisensor remote sensing images: multiapproach fusion of uncertain information. IEEE Trans Geosci Remote Sens, 46 (12): 4142–4152
Goshtasby A, Stockman G C, Page C V (1986). A region-based approach to digital image registration with subpixel accuracy. IEEE Trans Geosci Remote Sens, GE-24(3): 390–399
Greenfeld J S (2002). Matching GPS Observation to Location on a Digital Map. In: Proceedings of the 81st Annual Meeting of the Transportation Research Board, (3): 13
Harris C, Stephens M (1988). A combined corner and edge detector. In: Proceedings of Fourth Alvey Vision Conference,147–151
Hoyer P O (2004). Non-negative matrix factorization with sparseness constraints. J Mach Learn Res, 5: 1457–1469
Jiao W (2012). Free Viewpoint Action Recognition based on Selfsimilarities. In: Proceedings of the 11th International Conference on Signal Processing (ICSP), 2, 1131–1134
Ken C (2009). Efficient Retrieval of Deformable Shape Classes using Local Self-Similarities. In: Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops, 264–271
Kennedy J, Eberhart R C (1995). Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, 4, 1942–1948
Kennedy J, Eberhart R C (2001). Swarm Intelligence. San Francisco: Morgan Kaufmann Publisher
Kim J, Fessler J A (2004). Intensity-based image registration using robust correlation coefficients. IEEE Trans Med Imaging, 23(11): 1430–1444
Klein L A (2004). Sensor and Data Fusion: A Tool for Information Assessment and Decision Making. Bellingham: SPIE Press, 8–10
Lee H K, Kim T C (2012). Local self-similarity based backprojection for image upscaling. In: Proceedings of 2012 IEEE International Symposium on Circuits and Systems (ISCAS), 1215–1218
Li H, Manjunath B S, Mitra S K (1995). A contour-based approach to multisensor image registration. IEEE Trans Image Process, 4(3): 320–334
Liang J, Liu X, Huang K, Li X, Wang D, Wang X (2014). Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Trans Geosci Remote Sens, 52(1): 603–615
Liu S, Du X Y, Zhang J H (2009). Structure extracting and matching based on similarity-pictorial structure model for microscopic images. In: Proceedings of International Conference on Artificial Intelligence, 3: 181–185
Lowe D G (2004). Distinctive image features from scale-invariant key points. Int J Comput Vis, 60(2): 91–110
Meskine F, Mezouar M C E, Taleb N (2010). A rigid image registration based on the non subsampled contourlet transform and genetic algorithms. Sensors (Basel), 10(9): 8553–8571
Messerschmidt L, Engelbrecht A P (2004). Learning to play games using a PSO-based competitive learning approach. IEEE Trans Evol Comput, 8(3): 280–288
Pratt W K (1974). Correlation techniques of image registration. IEEE Trans Aerosp Electron Syst, AES-10(3): 353–358
Ricardo G (2012). Landmark localisation in brain MR images using feature point descriptors based on 3D local self-similarities. In: Proceedings of the 9th IEEE International Symposium on Biomedical Imaging, 1535–1538
Richards J A, Jia X (2006). Remote Sensing Digital Image Analysis (4th ed). Berlin: Springer-Verlag, 56–58
Sedaghat A, Ebadi H (2015). Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching. ISPRS J Photogramm Remote Sens, 108: 62–71
Shechtman E, Irani M (2007). Matching local self-similarities across images and videos. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1–8
Suri S, Reinartz P (2010). Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Trans Geosci Remote Sens, 48(2): 939–949
Taleb N, Bentoutou Y, Deforges O, Taleb A (2001). A 3-D space-time motion evaluation for image registration in digital subtraction angiography. Comput Med Imaging Graph, 25(3): 223–233
Viola P, Wells W M III (1997). Alignment by maximization of mutual information. Int J Comput Vis, 24(2): 137–154
Wachowiak M P, Smolikova R, Zheng Y, Zurada J M, Elmaghraby A S (2004). An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans Evol Comput, 8 (3): 289–301
Wolberg G, Zokai S (2000). Robust image registration using log-polar transform. In: Proceedings of IEEE International Conference on Image Processing, 1: 493–496
Wong A, Clausi D A (2007). ARRSI: automatic registration of remote sensing images. IEEE Trans Geosci Remote Sens, 45(5): 1483–1493
Yang H, Hou X (2012). Local self-similarity based texture classification. In: Proceedings of the 5th International Congress on Image and Signal Processing (CISP), 795–799
Yi Z, Chen Z, Yang X (2008). Multi-spectral remote image registration based on SIFT. Electron Lett, 44(2): 107–108
Zhang H G, Bai X, Zheng H X, Zhao H J, Zhou J, Cheng J, Lu H (2013). Hierarchical remote sensing image analysis via graph laplacian energy. IEEE Geosci Remote Sens Lett, 10(2): 396–400
Zheng H (2011). A novel approach for satellite image classification using local self-similarity. In: Proceedings of Geoscience and Remote Sensing Symposium (IGARSS), 2011 IEEE International, 2888‒2891
Zitová B, Flusser J (2003). Image registration methods: a survey. Image Vis Comput, 21(11): 977–1000
Acknowledgements
The work was supported by the National Natural Science Foundation of China (Grant No. 41371499) and the Natural Science Foundation of Guangdong Province (No. 2015A030313505).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, X., Chen, S., Zhuo, L. et al. Multi-sensor image registration by combining local self-similarity matching and mutual information. Front. Earth Sci. 12, 779–790 (2018). https://doi.org/10.1007/s11707-018-0717-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11707-018-0717-9