Abstract
Shadow removal, which converts images with shadow to images without shadow is currently an immature technology. When removing shadows, we need to consider various factors including illumination, environment etc. The traditional image processing method is complex and cumbersome, but the result is not ideal. Recently, with the rapid development of deep learning, various papers have proposed different methods to deal with this task, and the results have been greatly improved, but there are still some problems to be solved. At present, there are three problems in the shadow removal paper based on deep learning. First, the problem is color inconsistency. The shadow regions are difficult to restore the correct color and has a significant color difference from non-shadow regions after removal. Second, the shadow boundaries are clearly left in the image. Last, it is difficult to collect datasets in this field, resulting in a lack of training sets, making the model unable to adapt to various scenarios. We propose solutions to the above three problems respectively. The final model can effectively improve both the root mean square error (RMAE) and the structural similarity index (SSIM) of the shadow and non-shadow regions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Le, H., Samaras, D.: Shadow removal via shadow image decomposition. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp. 8577–8586 (2019). https://doi.org/10.1109/ICCV.2019.00867
Inoue, N., Yamasaki, T.: Learning from synthetic shadows for shadow detection and removal. IEEE Trans. Circuits Syst. Video Technol. 31(11), 4187–4197 (2021). https://doi.org/10.1109/TCSVT.2020.3047977
Xiao, C., She, R., Xiao, D., Ma, K.-L.: Fast Shadow removal using adaptive multi-scale illumination transfer: shadow removal by multi-scale illumination transfer. Comput. Graph. Forum 32(8), 207–218 (2013). https://doi.org/10.1111/cgf.12198
Guo, R., Dai, Q., Hoiem, D.: Paired regions for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2956–2967 (2013)
Qu, L., Tian, J., He, S., Tang, Y., Lau, R.W.: DeshadowNet: a multi-context embedding deep network for shadow removal. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 2308–2316 (2017). https://doi.org/10.1109/CVPR.2017.248
Goodfellow, I.J., et al.: Generative Adversarial Networks (2014). Accessed 04 June 2022. http://arxiv.org/abs/1406.2661
Hu, X., Jiang, Y., Fu, C.W., Heng, P.-A.: Mask-ShadowGAN: learning to remove shadows from unpaired data. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2472–2481 (2019). https://doi.org/10.1109/ICCV.2019.00256
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (2020). Accessed on 04 June 2022. http://arxiv.org/abs/1703.10593
Wang, J., Li, X., Hui, L., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal (2017). Accessed on 06 June 2022. http://arxiv.org/abs/1712.02478
Shor, Y., Lischinski, D.: The shadow meets the mask: pyramid-based shadow removal. Comput. Graph. Forum 27(2), 577–586 (2008). https://doi.org/10.1111/j.1467-8659.2008.01155.x
Cun, X., Pun, C.-M., Shi, C.: Towards ghost-free shadow removal via dual hierarchical aggregation network and shadow matting GAN (2019). Accessed on 04 June 2022. http://arxiv.org/abs/1911.08718
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks (2017). Accessed on 04 June 2022. http://arxiv.org/abs/1611.05431
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015). Accessed on 04 June 2022. http://arxiv.org/abs/1512.03385
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation (2015). Accessed on 05 June 2022. http://arxiv.org/abs/1505.04597
Zhang, H., et al.: ResNeSt: split-attention networks (2020). Accessed on 05 June 2022. http://arxiv.org/abs/2004.08955
Hu, X., Fu, C.-W., Zhu, L., Qin, J., Heng, P.-A.: Direction-aware spatial context features for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 42(11), 2795–2808 (2020). https://doi.org/10.1109/TPAMI.2019.2919616
Fu, L., et al.: Auto-exposure fusion for single-image shadow removal (2021). Accessed on 06 June 2022. http://arxiv.org/abs/2103.01255
Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., Jagersand, M.: BASNet: boundary-aware salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 7471–7481 (2019). https://doi.org/10.1109/CVPR.2019.00766
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Jin, Y., Sharma, A., Tan, R.T.: DC-ShadowNet: single-image hard and soft shadow removal using unsupervised domain-classifier guided network. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, pp. 5007–5016 (2021). https://doi.org/10.1109/ICCV48922.2021.00498
Zhu, L., et al.: Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 122–137. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_8
Acknowledgments
This research was partially supported in part by the Ministry of Science and Technology under contract numbers 111-2218-E-011 -011-MBK and 111-2221-E-011 -134 -, and also by the “Center for Cyber-physical System Innovation” from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Horng, SJ., Zhuang, CE. (2023). Deep Learning Based Shadow Removal: Target to Current Methodology Flaws. In: Arai, K. (eds) Intelligent Computing. SAI 2023. Lecture Notes in Networks and Systems, vol 711. Springer, Cham. https://doi.org/10.1007/978-3-031-37717-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-37717-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37716-7
Online ISBN: 978-3-031-37717-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)