Abstract
Visual odometry (VO), a method that estimates odometry using visual sensors, is hard to operate successfully with the low-resolution and noisy image sequences. To address this problem, a super-resolution technique is applied to input data before performing VO. Since most conventional super-resolution literature mainly deals with the resolution increment, we present a novel deep neural super-resolution network, which can remove noises as well. The execution time is also taken into account by adjusting the number of CNN layers for a real-time VO. By applying the proposed super-resolution approach, the resolution increases and noises disappear with a suitable speed, hence VO can be performed successfully. Experimental results show that the proposed method improves the VO performance compared with the conventional VO which uses low-resolution and noisy image sequences.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
D. Nistèr, O. Naroditsky, and J. Bergen, “Visual odometry,” Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. II, 2004.
E. Eade and T. Drummond, “Scalable monocular SLAM,” Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 469476, 2006.
A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, “MonoSLAM: real-time single camera slam,” IEEE Trans. on Pattern Analysis and Machine Intelligence, no. 6, pp. 10521067, 2007.
J. Civera, A. J. Davison, and J. M. Montiel, “Inverse depth parametrization for monocular SLAM,” IEEE Trans. on Robotics, vol. 24, no. 5, pp. 932945, 2008.
G. Klein and D. Murray, “Parallel tracking and mapping for small AR workspaces,” Proc. of IEEE and ACM International Symposium on Mixed and Augmented Reality (ASMAR), pp. 225234, 2007.
R. Mur-Artal and J. D. Tardós, “ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras,” IEEE Trans. on Robotics, vol. 33, no. 5, pp. 12551262, 2017.
R. Mur-Artal, J. M. M. Montiel, and J. D. Tardós, “ORB-SLAM: a versatile and accurate monocular slam system,” IEEE Trans. on Robotics, vol. 31, no. 5, pp. 11471163, 2015.
E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: an efficient alternative to SIFT or SURF,” Proc. of IEEE International Conf. Computer Vision (ICCV), pp. 25642571, 2011.
R. A. Newcombe, S. J. Lovegrove, and A. J. Davison, “DTAM: dense tracking and mapping in real-time,” Proc. of IEEE International Conf. Computer Vision (ICCV), pp. 23202327, 2011.
M. Pizzoli, C. Forster, and D. Scaramuzza, “REMODE: probabilistic, monocular dense reconstruction in real time,” Proc. of IEEE International Conf. Robotics and Automation (ICRA), pp. 26092616, 2014.
J. Engel, J. Sturm, and D. Cremers, “Semi-dense visual odometry for a monocular camera,” Proc. of IEEE International Conf. on Computer Vision (ICCV), pp. 14491456, 2013.
J. Engel, T. Schöps, and D. Cremers, “LSD-SLAM: large-scale direct monocular SLAM,” Proc. of the European Conference on Computer Vision, pp. 834849, 2014.
J. Engel, V. Koltun, and D. Cremers, “Direct sparse odometry,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 40, no. 3, pp. 611625, 2018.
X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. on Image Processing, vol. 10, no. 10, pp. 15211527, 2001.
L. Zhang and X. Wu, “An edge-guided image interpolation algorithm via directional filtering and data fusion,” IEEE Trans. on Image Processing, vol. 15, no. 8, pp. 22262238, 2006.
M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” Proceedings of British Machine Vision Conference, pp. 135.1–135.10, 2012.
X. Gao, K. Zhang, D. Tao, and X. Li, “Image superresolution with sparse neighbor embedding,” IEEE Trans. on Image Processing, vol. 21, no. 7, pp. 31943205, 2012.
J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, “Coupled dictionary training for image super-resolution,” IEEE Trans. on Image Processing, vol. 21, no. 8, pp. 34673478, 2012.
R. Timofte, V. De Smet, and L. Van Gool, “A+: adjusted anchored neighborhood regression for fast super-resolution,” Proc. of Asian Conf. on Computer Vision (ACCV), pp. 111126, 2014.
G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. on Graphics (TOG), vol. 30, no. 2, p. 12, 2011.
Z. Wang, Y. Yang, Z. Wang, S. Chang, J. Yang, and T. S. Huang, “Learning super-resolution jointly from external and internal examples,” IEEE Trans. on Image Processing, vol. 24, no. 11, pp.43594371, 2015.
C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” Proc. of European Conf. Computer Vision (ECCV), pp. 184199, 2014.
C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295307, 2016.
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation applied to hand-written zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541551, 1989.
J. Kim, J. Kwon Lee, and K. Mu Lee, “Accurate image super-resolution using very deep convolutional networks,” Proc. of IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 16461654, 2016.
J. Kim, J. K. Lee, and K. M. Lee, “Deeply-recursive convolutional network for image super-resolution,” Proc. of IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 16371645, 2016.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. of IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 770778, 2016.
C. Ledig, L. Theis, F. Huszàr, J. Caballero, A. Cunningham, A. Acosta, A. P. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” Proc. of IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 4681–4690, 2017.
B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, “Enhanced deep residual networks for single image super-resolution,” Proc. of IEEE Conf. Computer Vision and Pattern Recognition (CVPR) workshops, vol. 1, no. 2, pp. 136–144, 2017.
J. Engel, V. Usenko, and D. Cremers, “A photometrically calibrated benchmark for monocular visual odometry,” arXiv:1607.02555, July 2016.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Recommended by Associate Editor Hyun Myung under the direction of Editor Won-jong Kim. This work was supported in part by the Brain Korea 21 Plus Project and in part by a Bio-Mimetic Robot Research Center funded by Defense Acquisition Program Administration, and by Agency for Defense Development (UD190018ID).
Wonyeong Jeong received his B.S. degree in Electrical and Computer Engineering from Seoul National University, Seoul, the Republic of Korea in August 2013, where he is currently working toward a Ph.D. degree with the Department of Electrical and Computer Engineering. His major research interests include SLAM, vision-based robotics applications, multi-agent system coordination, and semantic scene understanding.
Jiyoun Moon received her Bachelors of Science in Robotics from Kwangwoon University in August 2014. Her major research interests include Natural language process, Semantic scene understanding, and Mission planning.
Beomhee Lee received his B.S. and M.S. degrees in Electronics Engineering from Seoul National University, Seoul, Korea in 1978 and 1980, respectively, and a Ph.D. degree in Computer Information, and control engineering from the University of Michigan, Ann Arbor, MI, USA in 1985. He was an Assistant Professor with the School of Electrical Engineering at Purdue University, West Lafayette, IN from 1985 to 1987. He joined Seoul National University in 1987, and is currently a Professor with the Department of Electrical and Computer Engineering. His research interests include multi-agent system coordination, control, and application. Prof. Lee has been a Fellow of the Robotics and Automation Society since 2004.
Rights and permissions
About this article
Cite this article
Jeong, W., Moon, J. & Lee, B. Error Improvement in Visual Odometry Using Super-resolution. Int. J. Control Autom. Syst. 18, 322–329 (2020). https://doi.org/10.1007/s12555-019-0256-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-019-0256-5