Abstract
Image super-resolution (SR) is one of the classic computer vision tasks. This paper proposes a super-resolution network based on adaptive frequency component upsampling, named SR-AFU. The network is composed of multiple cascaded dilated convolution residual blocks (CDCRB) to extract multi-resolution features representing image semantics, and multiple multi-size convolutional upsampling blocks (MCUB) to adaptively upsample different frequency components using CDCRB features. The paper also defines a new loss function based on the discrete wavelet transform, making the reconstructed SR images closer to human perception. Experiments on the benchmark datasets show that SR-AFU has higher peak signal to noise ratio (PSNR), significantly faster training speed and more realistic visual effects compared with the existing methods.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Freeman W T, Pasztor E C, Carmichael O T. Learning low-level vision. International Journal of Computer Vision, 2000, 40(1): 25–47
Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, de Marvao A M S M, Dawes T, O’Regan D, Rueckert D. Cardiac image superresolution with global correspondence using multi-atlas patchmatch. In: Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention. 2013, 9–16
Zhang S, Liang G, Pan S, Zheng L. A fast medical image super resolution method based on deep learning network. IEEE Access, 2018, 7: 12319–12327
Oh J, Lee C, Seo D C. Automated HRSI georegistration using orthoimage and SRTM: focusing KOMPSAT-2 imagery. Computers & Geosciences, 2013, 52: 77–84
Nogueira K, Penatti O A B, Dos Santos J A. Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recognition, 2017, 61: 539–556
Hu X, Ma P, Mai Z, Peng S, Yang Z, Wang L. Face hallucination from low quality images using definition-scalable inference. Pattern Recognition, 2019, 94: 110–121
Chen L C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 833–851
Li P, Wang Q, Zuo W, Zhang L. Log-Euclidean kernels for sparse representation and dictionary learning. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1601–1608
Lee Y, Choe Y. Neighbor embedding based single image superresolution using hybrid feature and adaptive weight decay regularization. In: Proceedings of the 4th IEEE International Conference on Consumer Electronics Berlin. 2014, 185–187
Dong C, Loy C C, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307
Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1646–1654
Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1637–1645
Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y. Residual dense network for image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 2472–2481
Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y. Image super-resolution using very deep residual channel attention networks. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 294–310
Rad M S, Bozorgtabar B, Marti U V, Basler M, Ekenel H K, Thiran J P. SROBB: targeted perceptual loss for single image super-resolution. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 2710–2719
Ng M K, Shen H, Lam E Y, Zhang L. A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP Journal on Advances in Signal Processing, 2007, 2007: 074585
Jiang K, Wang Z, Yi P, Jiang J. Hierarchical dense recursive network for image super-resolution. Pattern Recognition, 2020, 107: 107475
Dong C, Loy C C, Tang X. Accelerating the super-resolution convolutional neural network. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 391–407
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778
Lai W S, Huang J B, Ahuja N, Yang M H. Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5835–5843
Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 105–114
Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Loy C C. ESRGAN: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 2018, 63–79
Lim B, Son S, Kim H, Nah S, Lee K M. Enhanced deep residual networks for single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017, 1132–1140
Yu J, Fan Y, Yang J, Xu N, Wang Z, Wang X, Huang T. Wide activation for efficient and accurate image super-resolution. 2018, arXiv preprint arXiv: 1808.08718
Dai T, Cai J, Zhang Y, Xia S T, Zhang L. Second-order attention network for single image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 11057–11066
Zhao H, Gallo O, Frosio I, Kautz J. Loss functions for neural networks for image processing. 2018, arXiv preprint arXiv: 1511.08861
Timofte R, Agustsson E, Van Gool L, Yang M H, Zhang L, et al. NTIRE 2017 challenge on single image super-resolution: methods and results. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017, 1110–1121
Bevilacqua M, Roumy A, Guillemot C, Morel M L A. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of British Machine Vision Conference. 2012
Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations. In: Proceedings of the 7th International Conference on Curves and Surfaces. 2010, 711–730
Martin D, Fowlkes C, Tal D, Malik J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th IEEE International Conference on Computer Vision. 2001, 416–423
Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5197–5206
Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference for Learning Representations. 2015
Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600–612
Dong C, Loy C C, He K, Tang X. Learning a deep convolutional network for image super-resolution. In: Proceedings of the 13th European Conference on Computer Vision. 2014, 184–199
Tai Y, Yang J, Liu X, Xu C. MemNet: a persistent memory network for image restoration. In: Proceedings of IEEE International Conference on Computer Vision. 2017, 4549–4557
Haris M, Shakhnarovich G, Ukita N. Deep back-projection networks for super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 1664–1673
Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W. Feedback network for image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 3862–3871
Hui Z, Gao X, Yang Y, Wang X. Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM International Conference on Multimedia. 2019, 2024–2032
Zhao H, Kong X, He J, Qiao Y, Dong C. Efficient image superresolution using pixel attention. In: Proceedings of the European Conference on Computer Vision. 2020, 56–72
Wang C, Li Z, Shi J. Lightweight image super-resolution with adaptive weighted learning network. 2019, arXiv preprint arXiv: 1904.02358
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant Nos. 61603197 and 61772284), Natural Science Foundation of Nanjing University of Posts and Telecommunications (NY221071).
Author information
Authors and Affiliations
Corresponding author
Additional information
Ke-Jia Chen is an associate professor in Nanjing University of Posts and Telecommunications, China. She received her PhD in Université de Technologie de Compiègne, France and her master’s degree in Nanjing University, China. She joined Jiangsu Key Laboratory of Big Data Security & Intelligent Processing, China in 2017. Her current research focuses on machine learning and its applications in complex network analysis.
Mingyu Wu received the BS degree in Electronic Information Engineering from Nanjing University of Posts and Telecommunications, China in 2021. He is working toward the master degree in Signal and information processing at Nanjing University of Posts and Telecommunications, China. His current research interests include casual inference, sequence modeling and cross-modal analysis.
Yibo Zhang received the BS degree in Electronic Information Engineering from Nanjing University of Posts and Telecommunications, China in 2021. His research interests include computer vision, UAV system and automatic driving.
Zhiwei Chen received the BS degree in Electronic Information Engineering from Nanjing University of Posts and Telecommunications, China in 2021. He is working toward the master degree in Electronic information at the South China Normal University, China. His current research interests include machine learning, computer vision and Neuromorphic computation.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Chen, KJ., Wu, M., Zhang, Y. et al. SR-AFU: super-resolution network using adaptive frequency component upsampling and multi-resolution features. Front. Comput. Sci. 17, 171307 (2023). https://doi.org/10.1007/s11704-021-0562-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-021-0562-y