Skip to main content
SpringerLink
Log in

Super-Resolution with Deep Learning Techniques: A Review

  • Chapter
  • First Online:
Computational Intelligence Methods for Super-Resolution in Image Processing Applications

Abstract

Currently the subject of research in remote sensing and computer vision practices is the deep learning neural network. The super-resolution (SR) technique is an image remastering method that reproduces a high-quality image from a low-resolution (LR) image. It has various applications in different fields such as autonomous driving systems, analyzing footage of security, and medical imaging in health care. Currently super-resolution technologies have been proliferating before the introduction of deep learning (DL) techniques like generative adversarial networks (GANs) and convolutional neural networks (CNNs). Super-resolution based on deep learning is attempting to find out that traditional algorithm-based upscaling strategies lack fine detail and cannot remove compression artifacts and defects. Undoubtedly new methods in SR, based on deep learning techniques, have significantly increased the level of detail in high-quality images that they generate compared to earlier technologies. Interpolation is the most commonly used technique for upscaling a picture. Even though SRCNN is now superior to standard techniques, the perceptual loss is a potential solution which helps in minimizing mean squared error (MSE). Although the photographs generated by SISR are of higher resolution, they are often blurred, lack high-frequency, fine-grained details, and look dull in comparison to truly high-quality photographs. To build a model capable of creating more effectively satisfying and less blurry photographs, a model is used that can capture the perceptual differences between the original image and the generated one. Super-resolution with generative adversarial networks can be used to achieve this, which constructs high-quality photographs by applying a combination of an adversarial network and a deep network. This chapter intends to provide a review on super-resolution from the point of deep learning methods and their applications. It also includes the main contributions in recent years and discusses future research and challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhu, X. (2014). Computational intelligence techniques and applications. Computational Intelligence Techniques in Earth and Environmental Sciences, 3–26. https://doi.org/10.1007/978-94-017-8642-3_1.

  2. 302 Found. (2019). Retrieved from https://towardsdatascience.com/deep-learning-based-super-resolution-without-using-a-gan-11c9bb5b6cd5

  3. Morera-Delfín, L., Pinto-Elías, R., & Ochoa-Domínguez, H.-J. (2018). Overview of super-resolution techniques. Advanced Topics on Computer Vision, Control and Robotics in Mechatronics, 101–127. https://doi.org/10.1007/978-3-319-77770-2_5.

  4. Deep Learning based image Super-Resolution to enhance photos. (2018, July 25). Retrieved from https://cv-tricks.com/deep-learning-2/image-super-resolution-to-enhance-photos/

  5. Wang, Z., Chen, J., & Hoi, S. C. H. (2020). Deep learning for image super-resolution: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1. https://doi.org/10.1109/tpami.2020.2982166.

  6. Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016). Image super-resolution: The techniques, applications, and future. Signal Processing, 128, 389–408. https://doi.org/10.1016/j.sigpro.2016.05.002.

    Article  Google Scholar 

  7. Goyal, R. (2018, May 9). Five important techniques that you should know about deep learning [Blog post]. Retrieved from https://www.zeolearn.com/magazine/five-important-techniques-that-you-should-know-about-deep-learning

  8. Ramavat, K., Joshi, M., & Swadas, P. B. (2016). A survey of super-resolution techniques. International Research Journal of Engineering and Technology, 3(12), 1035–1039. Retrieved from https://www.irjet.net/archives/V3/i12/IRJET-V3I12238.pdf.

    Google Scholar 

  9. Zhang, Y., & Xiang, Y. (2018). Recent advances in deep learning for single image super-resolution. Advances in Brain Inspired Cognitive Systems, 85–95. https://doi.org/10.1007/978-3-030-00563-4_9.

  10. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., et al. (2017). Photo-realistic single image super-resolution using a generative adversarial network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 105–114. https://doi.org/10.1109/cvpr.2017.19.

    Article  Google Scholar 

  11. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. C. (2017). Improved training of Wasserstein GANs. ArXiv, abs/1704.00028.

    Google Scholar 

  12. Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein gan. arXiv preprint arXiv:1701.07875.

    Google Scholar 

  13. Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2018). Progressive growing of GANs for improved quality, stability, and variation. ArXiv, abs/1710.10196.

    Google Scholar 

  14. Denton, E. L., Chintala, S., Szlam, A., & Fergus, R. (2015). Deep generative image models using a Laplacian pyramid of adversarial networks. ArXiv, abs/1506.05751.

    Google Scholar 

  15. Im, D.J., Kim, C.D., Jiang, H., & Memisevic, R. (2016). Generating images with recurrent adversarial networks. ArXiv, abs/1602.05110.

    Google Scholar 

  16. Park, S. C., Park, M. K., & Kang, M. G. (2003). Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine, 20(3), 21–36. https://doi.org/10.1109/msp.2003.1203207.

    Article  Google Scholar 

  17. Nasrollahi, K., & Moeslund, T. B. (2014). Super-resolution: A comprehensive survey. Machine Vision and Applications, 25(6), 1423–1468. https://doi.org/10.1007/s00138-014-0623-4.

    Article  Google Scholar 

  18. Tian, J., & Ma, K.-K. (2011). A survey on super-resolution imaging. Signal, Image and Video Processing, 5(3), 329–342. https://doi.org/10.1007/s11760-010-0204-6.

    Article  Google Scholar 

  19. Van Ouwerkerk, J. D. (2006). Image super-resolution survey. Image and Vision Computing, 24(10), 1039–1052. https://doi.org/10.1016/j.imavis.2006.02.026.

    Article  Google Scholar 

  20. Yang, C.-Y., Ma, C., & Yang, M.-H. (2014). Single-image super-resolution: A benchmark. Computer Vision – ECCV, 2014, 372–386. https://doi.org/10.1007/978-3-319-10593-2_25.

    Article  Google Scholar 

  21. Thapa, D., Raahemifar, K., Bobier, W. R., & Lakshminarayanan, V. (2016). A performance comparison among different super-resolution techniques. Computers & Electrical Engineering, 54, 313–329. https://doi.org/10.1016/j.compeleceng.2015.09.011.

    Article  Google Scholar 

  22. Kańska, K. (2019, April 25). Cookie and Privacy Settings [Blog post]. Retrieved from https://deepsense.ai/using-deep-learning-for-single-image-super-resolution/

  23. Salaria, S. (2019, August 22). Using the super-resolution convolutional neural network for image restoration [Blog post]. Retrieved from https://medium.com/datadriveninvestor/using-the-super-resolution-convolutional-neural-network-for-image-restoration-ff1e8420d846

  24. Razmjooy, N., Estrela, V. V., & Loschi, H. J. (2020). Entropy-based breast cancer detection in digital mammograms using world cup optimization algorithm. International Journal of Swarm Intelligence Research (IJSIR), 11(3), 1–18.

    Article  Google Scholar 

  25. Raj, B. (2019, July 1). An Introduction to Super-Resolution using Deep Learning [Blog post]. Retrieved from https://medium.com/beyondminds/an-introduction-to-super-resolution-using-deep-learning-f60aff9a499d

  26. Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. https://doi.org/10.1109/tip.2003.819861.

    Article  Google Scholar 

  27. 302 Found. (2019, June 1). Retrieved from https://heartbeat.fritz.ai/research-guide-image-quality-assessment-c4fdf247bf89

  28. Mahmoudpour, S., & Kim, M. (2015). A study on the relationship between depth map quality and stereoscopic image quality using upsampled depth maps. Emerging Trends in Image Processing, Computer Vision and Pattern Recognition, 149–160. https://doi.org/10.1016/b978-0-12-802045-6.00010-7.

  29. Sheikh, H. R., & Bovik, A. C. (2006). Image information and visual quality. IEEE Transactions on Image Processing, 15(2), 430–444. https://doi.org/10.1109/tip.2005.859378.

    Article  Google Scholar 

  30. Tsai, C., Liu, H., & Tasi, M. (2011). Design of a scan converter using the cubic convolution interpolation with canny edge detection. In Proceedings of the international conference on electric information and control engineering (pp. 5813–5816).

    Google Scholar 

  31. de Jesus, et al. (2020, April). Using Transmedia Approaches in STEM. In 2020 IEEE Global Engineering Education Conference (EDUCON) (pp. 1013–1016). IEEE.

    Google Scholar 

  32. Sheikh, H. R., Sabir, M. F., & Bovik, A. C. (2006). A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing, 15(11), 3440–3451. https://doi.org/10.1109/tip.2006.881959.

    Article  Google Scholar 

  33. Zhou, W., & Bovik, A. C. (2009). Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE Signal Processing Magazine, 26(1), 98–117. https://doi.org/10.1109/msp.2008.930649.

    Article  Google Scholar 

  34. Wang, X., Yu, K., Dong, C., & Change Loy, C. (2018). Recovering realistic texture in image super-resolution by deep spatial feature transform. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, 606–615. https://doi.org/10.1109/cvpr.2018.00070.

    Article  Google Scholar 

  35. Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015). Deep networks for image super-resolution with sparse prior. In 2015 IEEE International Conference on Computer Vision (ICCV) (pp. 370–378). https://doi.org/10.1109/iccv.2015.50.

    Chapter  Google Scholar 

  36. Xu, X., Sun, D., Pan, J., Zhang, Y., Pfister, H., & Yang, M.-H. (2017). Learning to super-resolve blurry face and text images. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 251–260). https://doi.org/10.1109/iccv.2017.36.

    Chapter  Google Scholar 

  37. Dahl, R., Norouzi, M., & Shlens, J. (2017). Pixel recursive super resolution. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 5449–5458). https://doi.org/10.1109/iccv.2017.581.

    Chapter  Google Scholar 

  38. Lai, W.-S., Huang, J.-B., Ahuja, N., & Yang, M.-H. (2019). Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(11), 2599–2613. https://doi.org/10.1109/tpami.2018.2865304.

    Article  Google Scholar 

  39. Sajjadi, M. S. M., Scholkopf, B., & Hirsch, M. (2017). EnhanceNet: Single image super-resolution through automated texture synthesis. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 4501–4510). https://doi.org/10.1109/iccv.2017.481.

    Chapter  Google Scholar 

  40. Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2003). Multiscale structural similarity for image quality assessment. The Thirty-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, 1398–1402. https://doi.org/10.1109/acssc.2003.1292216.

    Article  Google Scholar 

  41. Mittal, A., Soundararajan, R., & Bovik, A. C. (2013). Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 20(3), 209–212. https://doi.org/10.1109/lsp.2012.2227726.

    Article  Google Scholar 

  42. Lin, Z., Zhang, L., Mou, X., & Zhang, D. (2011). FSIM: A feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 20(8), 2378–2386. https://doi.org/10.1109/tip.2011.2109730.

    Article  MathSciNet  MATH  Google Scholar 

  43. Blau, Y., & Michaeli, T. (2018). The perception-distortion Tradeoff. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, 6228–6237. https://doi.org/10.1109/cvpr.2018.00652.

    Article  Google Scholar 

  44. Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. Computer Vision – ECCV, 2016, 694–711. https://doi.org/10.1007/978-3-319-46475-6_43.

    Article  Google Scholar 

  45. Gatys, L. A., Ecker, A. S., & Bethge, M. (2016). Image style transfer using convolutional neural networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 2414–2423. https://doi.org/10.1109/cvpr.2016.265.

    Article  Google Scholar 

  46. Perceptual Loss Functions. (2019, May 17). Retrieved from https://deepai.org/machine-learning-glossary-and-terms/perceptual-loss-function

  47. Super-Resolution Deep Learning: Making the Future Clearer. [Blog Post]. Retrieved from https://missinglink.ai/guides/computer-vision/super-resolution-deep-learning-making-future-clearer/

  48. Yashwanth, N., Navya, P., Rukhiya, M., Prasad, K. S., & Deepthi, K. S. (2019). Survey on generative adversarial networks. International Journal of Advance Research, Ideas and Innovations in Technology, 5, 239–244.

    Google Scholar 

  49. An Evolution in Single Image Super-Resolution using Deep Learning. (2019, December 3). Retrieved from https://towardsdatascience.com/an-evolution-in-single-image-super-resolution-using-deep-learning-66f0adfb2d6b

  50. Sinha, V. (2019, December 17). Super Resolution GAN (SRGAN) [Blog post]. Retrieved from https://medium.com/analytics-vidhya/super-resolution-gan-srgan-5e10438aec0c

  51. Hui, J. (2018, July 2). GAN — Super Resolution GAN (SRGAN) [Blog post]. Retrieved from https://medium.com/@jonathan_hui/gan-super-resolution-gan-srgan-b471da7270ec

  52. Shaikh, F. (2020, May 11). Top 5 Interesting Applications of GANs for Every Machine Learning Enthusiast! Retrieved from https://www.analyticsvidhya.com/blog/2019/04/top-5-interesting-applications-gans-deep-learning/

  53. Gonog, L., & Zhou, Y. (2019). A review: Generative adversarial networks. 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA), 505–510.

    Google Scholar 

  54. Monteiro, et al. (2018). Health 4.0: applications, management, technologies and review. Personalized Medicine, 2(4), 262–276.

    Google Scholar 

  55. Razmjooy, N., et al. (2020). Computer-aided diagnosis of skin cancer: A review. Current Medical Imaging, 16(7), 781–793.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lovely Professional University, Phagwara, India

    Aarti

  2. Dr B R Ambedkar National Institute of Technology, Jalandhar, India

    Amit Kumar

Authors
  1. Aarti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amit Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. SAEF’s Angadi Institute of Technology and Management, Belagavi, Karnataka, India

    Anand Deshpande

  2. Electrical & Computer Engineering, Universidade Federal Fluminese, Duque de Caxias, Rio de Janeiro, Brazil

    Vania V. Estrela

  3. Amirkabir University of Technology, Ghent, Belgium

    Navid Razmjooy

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Aarti, Kumar, A. (2021). Super-Resolution with Deep Learning Techniques: A Review. In: Deshpande, A., Estrela, V.V., Razmjooy, N. (eds) Computational Intelligence Methods for Super-Resolution in Image Processing Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-67921-7_3

Download citation

Publish with us

Policies and ethics

Search

Navigation