Abstract
A geo-localization method is proposed for military and civilian applications, which is used when no global navigation satellite system (GNSS) information is available. The open graphics library (OpenGL) is used to build a three-dimensional geographic model of the test area using digital elevation model (DEM) data, and the skyline can thus be extracted with the model to form a database. Then, MultiSkip DeepLab (MS-DeepLab), a fully convolutional semantic segmentation network with multiple skip structures, is proposed to extract the skyline from the query image. Finally, a matching model based on convolutional neural network (CNN) feature is adopted to calculate the similarity between the skyline features of the query image and the DEM database to realize automatic geo-localization. The experiments are conducted at a 202.6 km2 test site in north-eastern Changsha, China. 50 test points are selected to verify the effectiveness of the proposed method, and an excellent result with an average positioning error of 49.29 m is obtained.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
TALLURI R, AGGARWAL J. Position estimation for an autonomous mobile robot in an outdoor environment[J]. IEEE transactions on robotics and automation, 1992, 8(5): 573–584.
STEIN F, MEDIONI G. Map-based localization using the panoramic horizon[J]. IEEE transactions on robotics and automation, 1996, 11(6): 892–896.
NAVAL P C. Camera pose estimation by alignment from a single mountain image[EB/OL]. (2010-07) [2021-10-11]. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=87E95F9B1E103A9679DA9009-C4CF1-F74?doi=10.1.1.102.9949&rep=rep1&type=pdf.
NAVAL P C, MUKUNOKI M, MINOH M, et al. Estimating camera position and orientation from geographical map and mountain image[EB/OL]. (1997-04) [2021-10-11]. http://citeseerx.ist.psu.edu/viewdoc/Download?doi=10.1.1.14.3619&rep=rep1&type=pdf.
WOO J, SON K, LI T, et al. Vision-based UAV navigation in mountain area[C]//Proceedings of the IAPR Conference on Machine Vision Applications (IAPR MVA), May 16–18, 2007, Tokyo, Japan. 2007: 236–239.
BAATZ G, SAURER O, KOSER K, et al. Large scale visual geo-localization of images in mountainous terrain[C]//Proceedings of the 12th European Conference on Computer Vision, October 7–13, 2012, Florence, Italy. Berlin, Heidelberg: Springer-Verlag, 2012: 517–530.
TZENG E, ZHAI A, CLEMENTS M, et al. User-driven geolocation of untagged desert imagery using digital elevation models[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 23–28, 2013, Portland, OR, USA. New York: IEEE, 2013: 237–244.
PORZI L, BULO S R, VALIGI P, et al. Learning contours for automatic annotations of mountains pictures on smartphone[C]//Proceedings of the International Conference on Distributed Smart Cameras, September 5, 2014, California, USA. New York: ACM, 2014: 131–136.
BABOUD L, CADIK M, EISEMANN E, et al. Automatic photo-to-terrain alignment for the annotation of mountain picture[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition, June 20–25, 2011, Colorado Springs, Colorado, USA. New York: IEEE, 2011: 41–48.
HAMMOUD R I, KUZDEBA S A, BERARD B, et al. Overhead-based image and video geo-localization framework[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 23–28, 2013, Portland, OR, USA. New York: IEEE, 2013: 320–327.
CHEN Y, QIAN G, GUNDA K, et al. Camera geolocation from mountain images[C]//Proceedings of the 18th International Conference on Information Fusion, July 6–9, 2015, Washington, DC, USA. New York: IEEE, 2015: 1587–1596.
SAURER O, BAATZ G, KOSER K, et al. Image based geo-localization in the Alps[J]. International journal of computer vision, 2016, 116(3): 213–225.
GRELSSON B, ROBINSON A, FELSBERG M, et al. GNSS-level accurate camera localization with HorizonNet[J]. Journal of field robotics, 2020, 37(6): 951–971.
CHIODINI S, PERTILE M, DEBEI S, et al. Mars rovers localization by matching local horizon to surface digital elevation models[C]//Proceedings of the IEEE International Workshop on Metrology for AeroSpace, June 21–23, 2017, Padua, Italy. New York: IEEE, 2017.
FUKUDA S, NAKATANI S, NISHIYAMA M, et al. Geo-localization using ridgeline features extracted from 360-degree images of sand dunes[C]//Proceedings of the 15th International Conference on Computer Vision Theory and Applications, February 27–29, 2020, Valletta, Malta. New York: IEEE, 2020: 621–627.
DANIEL S T, LI M, MARGARET H. Face recognition: from traditional to deep learning methods[EB/OL]. (2019-11-15) [2021-10-11]. https://arxiv.org/pdf/1811.00116.pdf.
RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, October 5–9, 2015, Munich, Germany. Berlin, Heidelberg: Springer-Verlag, 2015: 234–241.
CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision, September 8–14, 2018, Munich, Germany. Berlin, Heidelberg: Springer-Verlag, 2018: 833–851.
LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 7–12, 2015, Boston, MA, USA. New York: IEEE, 2015: 3431–3440.
CHEN L C, PAPANDEOU G, KOKKINOS I, et al. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40: 834–848.
SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2014-07-15) [2021-10-11]. https://arxiv.org/pdf/1409.1556v6.pdf.
BREJCHA J, CADIK M. State-of-the-art in visual geo-localization[J]. Pattern analysis and applications, 2017, 20: 613–637.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been supported by the National Natural Science Foundation of China (No.61502537), and the Science & Technology Innovation System for National Defense of China (Nos.17-163-11-ZT-002-032-01 and 193-A11-103-11-04).
Statements and Declarations
The authors declare that there are no conflicts of interest related to this article.
Rights and permissions
About this article
Cite this article
Tang, J., Gong, C., Guo, F. et al. Geo-localization based on CNN feature matching. Optoelectron. Lett. 18, 300–306 (2022). https://doi.org/10.1007/s11801-022-1148-0
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11801-022-1148-0