Abstract
In this work, we present an AI-based Augmented Reality (AR) system for indoor planning and refurbishing applications. AR can be an important medium for such applications, as it facilitates more effective concept conveyance and additionally acts as an efficient and immediate designer-to-client communication channel. However, since AR only overlays, and cannot replace or remove, our system relies on Diminished Reality (DR) to support deployment to real-world already furnished indoor scenes. Further, and contrary to the traditional mobile AR application approach, our system offers on-demand Virtual Reality (VR) viewing, relying on spherical (360°) panoramas, capitalizing on their user-friendliness for indoor scene capturing. Given that our system is an integration of different AI services, we analyze its performance differentials concerning the components comprising it. This analysis is both quantitative and qualitative, with the latter realized through user surveys, and provides a complete systemic assessment of an attempt for a user-facing, automatic AR/DR system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Torchserve (2021). https://github.com/pytorch/serve
Gkitsas, V., Sterzentsenko, V., Zioulis, N., Albanis, G., Zarpalas, D.: PanoDR: spherical panorama diminished reality for indoor scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3716–3726
Chen, L.-C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation (2017). arXiv:1706.05587
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Herling, J., Broll, W.: Advanced self-contained object removal for realizing real-time diminished reality in unconstrained environments. In: 2010 IEEE International Symposium on Mixed and Augmented Reality, pp. 207–212. IEEE (2010)
Jam, J., Kendrick, C., Walker, K., Drouard, V., Hsu, J.G.-S., Yap, M.H.: A comprehensive review of past and present image inpainting methods. Comput. Vision Image Understand. 103147 (2020)
Jiddi, S., Pugh, B., Dai, Q., Puig, L., Lianos, N., Gauthier, P., Totty, B., Dorbie, A., Yin, J., Wong, K.: An end-to-end mixed reality product for interior home furnishing. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (2020)
Kawai, N., Sato, T., Yokoya, N.: Diminished reality based on image inpainting considering background geometry. IEEE Trans. Visual Comput. Graphics 22(3), 1236–1247 (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv:1412.6980
Mori, S., Ikeda, S., Saito, H.: A survey of diminished reality: techniques for visually concealing, eliminating, and seeing through real objects. IPSJ Trans. Comput. Vision Appl. 9(1), 1–14 (2017)
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: an imperative style, high-performance deep learning library (2019). arXiv:1912.01703
Phan, V.T., Choo, S.Y.: Interior design in augmented reality environment. Int. J. Comput. Appl. 5(5), 16–21 (2010)
Queguiner, G., Fradet, M., Rouhani, M.: Towards mobile diminished reality. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 226–231. IEEE (2018)
Rameau, F., Ha, H., Joo, K., Choi, J., Park, K., Kweon, I.S.: A realtime augmented reality system to see-through cars. IEEE Trans. Visual Comput. Graphics 22(11), 2395–2404 (2016)
Reuksupasompon, P., Aruncharathorn, M., Vittayakorn, S.: Ar development for room design. In: 2018 15th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 1–6. IEEE (2018)
Schmalstieg, D., Mori, S., Erat, O., Kalkofen, D., Broll, W., Saito, H.: Inpaintfusion: incremental rgb-d inpainting for 3d scenes. In IEEE Trans. Visual. Comput. Graph. (2020)
Siltanen, S.: Diminished reality for augmented reality interior design. Vis. Comput. 33(2), 193–208 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv:1409.1556
Sun, C., Hsiao, C., Sun, M., Chen, H.: Horizonnet: learning room layout with 1d representation and pano stretch data augmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16–20, 2019, pp. 1047–1056 (2019)
Wong, K., Jiddi, S., Alami, Y., Guindi, P., Totty, B., Guo, Q., Otrada, M., Gauthier, P.: Exploiting arkit depth maps for mixed reality home design. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (2020)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586–595 (2018)
Zhao, H., Kong, X., He, J., Qiao, Y., Dong, C.: Efficient image superresolution using pixel attention (2020). arXiv:2010.01073
Zheng, J., Zhang, J., Li, J., Tang, R., Gao, S., Zhou, Z.: Structured3d: a large photo-realistic dataset for structured 3d modeling. In: Proceedings of The European Conference on Computer Vision (ECCV) (2020)
Zhu, P., Abdal, R., Qin, Y., Wonka, P.: Sean: image synthesis with semantic region-adaptive normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5104–5113 (2020)
Acknowledgements
We thank Werner Bailer (Joanneum Research), Georg Thallinger (Joanneum Research), Vladimiros Sterzentsenko (Information Technologies Institute/Centre for Research and Technology), and Suzana Farokhian (Usability Partners) for insightful discussion and feedback. This work was supported by the EC funded H2020 project ATLANTIS [GA 951900].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Albanis, G. et al. (2023). An AI-Based System Offering Automatic DR-Enhanced AR for Indoor Scenes. In: Nakamatsu, K., Patnaik, S., Kountchev, R., Li, R., Aharari, A. (eds) Advanced Intelligent Virtual Reality Technologies. Smart Innovation, Systems and Technologies, vol 330. Springer, Singapore. https://doi.org/10.1007/978-981-19-7742-8_15
Download citation
DOI: https://doi.org/10.1007/978-981-19-7742-8_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-7741-1
Online ISBN: 978-981-19-7742-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)