Abstract
Single-view 3D shape reconstruction is an ill-posed problem that poses a major challenge to the field of Computer Vision. With the advent of the deep learning era, many methods are being continuously proposed to tackle this problem because of its importance. This paper proposes a data-driven method for single-view mesh-based 3D reconstruction and generation. Specifically, it presents a framework that is capable of reconstructing shapes and generating novel variants of these shapes from a single RGB image. The proposed framework consists of a mesh reconstruction network and a mesh autoencoder network. The framework is trained in two stages on synthetic data with 3D supervision. The quantitative and qualitative evaluations show promising results in the direction of solving this long-standing problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Schonberger, J.L., Frahm, J.-M.: Structure-from-motion revisited. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104–4113. IEEE, Las Vegas (2016)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR 2006), pp. 519–528. IEEE, New York (2006)
Prasad, M., Fitzgibbon, A.: Single view reconstruction of curved surfaces. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR 2006), pp. 1345–1354. IEEE, New York (2006)
Fahim, G., Amin, K., Zarif, S.: Single-view 3D reconstruction: a survey of deep learning methods. Comput. Graph. 94, 164–190 (2021)
Choy, C.B., Xu, D., Gwak, J., Chen, K., Savarese, S.: 3D-R2N2: a unified approach for single and multi-view 3D object reconstruction. In: Computer Vision – ECCV 2016, pp. 628–644. Springer, Cham (2016)
Girdhar, R., Fouhey, D.F., Rodriguez, M., Gupta, A.: Learning a predictable and generative vector representation for objects. In: Computer Vision – ECCV 2016, pp. 484–499. Springer, Cham (2016)
Wu, J., Zhang, C., Xue, T., Freeman, W.T., Tenenbaum, J.B.: Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 82–90. Curran Associates Inc., Barcelona (2016)
Fan, H., Su, H., Guibas, L.: A point set generation network for 3D object reconstruction from a single image. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2463–2471. IEEE, Honolulu (2017)
Mandikal, P., Navaneet, K.L., Agarwal, M., Venkatesh Babu, R.: 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image. arXiv e-prints. 1807. arXiv:1807.07796 (2018)
Sun, R., Gao, Y., Fang, Z., Wang, A., Zhong, C.: SSL-net: point-cloud generation network with self-supervised learning. IEEE Access 7, 82206–82217 (2019)
Lu, Q., Xiao, M., Lu, Y., Yuan, X., Yu, Y.: Attention-based dense point cloud reconstruction from a single image. IEEE Access 7, 137420–137431 (2019)
Afifi, A.J., Magnusson, J., Soomro, T.A., Hellwich, O.: Pixel2Point: 3D object reconstruction from a single image using CNN and initial sphere. IEEE Access 9, 110–121 (2021)
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., Jiang, Y.-G.: Pixel2Mesh: generating 3D mesh models from single RGB images. In: Computer Vision – ECCV 2018, pp. 55–71. Springer, Cham (2018)
Smith, E., Fujimoto, S., Romero, A., Meger, D.: GEOMetrics: exploiting geometric structure for graph-encoded objects. In: Proceedings of the 36th International Conference on Machine Learning, pp. 5866–5876. PMLR (2019)
Kato, H., Ushiku, Y., Harada, T.: Neural 3D mesh renderer. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3907–3916. IEEE, Salt Lake City (2018)
Liu, S., Chen, W., Li, T., Li, H.: Soft rasterizer: a differentiable renderer for image-based 3D reasoning. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7707–7716. IEEE, Seoul (2019)
Chen, W., Ling, H., Gao, J., Smith, E., Lehtinen, J., Jacobson, A., Fidler, S.: Learning to predict 3D objects with an interpolation-based differentiable renderer. In: Advances in Neural Information Processing Systems, pp. 9605–9616. Curran Associates, Inc. (2019)
Gong, S., Chen, L., Bronstein, M., Zafeiriou, S.: SpiralNet++: a fast and highly efficient mesh convolution operator. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 4141–4148. IEEE, Seoul (2019)
Ranjan, A., Bolkart, T., Sanyal, S., Black, M.J.: Generating 3D faces using convolutional mesh autoencoders. In: Computer Vision – ECCV 2018, pp. 725–741. Springer, Cham (2018)
Garland, M., Heckbert, P.S.: Surface simplification using quadric error metrics. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH 1997, pp. 209–216. ACM Press (1997)
Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., Xiao, J., Yi, L., Yu, F.: ShapeNet: An Information-Rich 3D Model Repository. arXiv:1512.03012 [cs] (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fahim, G., Amin, K., Zarif, S. (2021). Single-View 3D Mesh Reconstruction and Generation. In: Hassanien, A.E., et al. Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2021). AICV 2021. Advances in Intelligent Systems and Computing, vol 1377. Springer, Cham. https://doi.org/10.1007/978-3-030-76346-6_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-76346-6_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-76345-9
Online ISBN: 978-3-030-76346-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)