Abstract
The creation of surfaces from overlapping images taken from different vantages is a hard and important problem in computer vision. Recent developments fall primarily into two categories: the use of dense matching to produce point clouds from which surfaces are built, and the construction of surfaces from images directly. This paper presents a new method for surface reconstruction falling in the second category. First, a strongly motivated variational framework is built from the ground up based on a limiting case of photo-consistency. The framework includes a powerful new edge preserving smoothness term and exploits the input images exhaustively, directly yielding high quality surfaces instead of dealing with issues (such as noise or misalignment) after the fact. Numeric solution is accomplished with a combination of Gauss-Newton descent and the finite element method, yielding deep convergence in few iterates. The method is fast, robust, very insensitive to view/scene configurations, and produces state-of-the-art results in the Middlebury evaluation.
Chapter PDF
Similar content being viewed by others
References
Bookstein, F.L.: Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(6), 567–585 (1989)
Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM Journal on Imaging Sciences 3(3), 492–526 (2010)
Brenner, S.C., Scott, R.: The mathematical theory of finite element methods, vol. 15. Springer (2008)
Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 41–48. IEEE (2009)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Desbrun, M., Meyer, M., Schröder, P., Barr, A.H.: Implicit fairing of irregular meshes using diffusion and curvature flow. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 317–324. ACM Press/Addison-Wesley Publishing Co. (1999)
Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. In: ACM Transactions on Graphics (TOG), vol. 27, p. 67. ACM (2008)
Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. In: ACM Transactions on Graphics (TOG), vol. 21, pp. 249–256. ACM (2002)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(8), 1362–1376 (2010)
Garland, M., Heckbert, P.S.: Fast polygonal approximation of terrains and height fields. School of Computer Science, Carnegie Mellon University (1995)
Gay-Bellile, V., Bartoli, A., Sayd, P.: Direct estimation of nonrigid registrations with image-based self-occlusion reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1), 87–104 (2010)
Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011)
Goldman, R.: Curvature formulas for implicit curves and surfaces. Computer Aided Geometric Design 22(7), 632–658 (2005)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press (2003)
Hartley, R.I., Sturm, P.: Triangulation. Computer vision and image understanding 68(2), 146–157 (1997)
Hernández, C., Vogiatzis, G., Cipolla, R.: Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(3), 548–554 (2008)
Hiep, V.H., Keriven, R., Labatut, P., Pons, J.P.: Towards high-resolution large-scale multi-view stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1430–1437. IEEE (2009)
Hirschmuller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 807–814. IEEE (2005)
Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing (2006)
Kobbelt, L., Campagna, S., Vorsatz, J., Seidel, H.P.: Interactive multi-resolution modeling on arbitrary meshes. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 105–114. ACM (1998)
Kolev, K., Klodt, M., Brox, T., Cremers, D.: Continuous global optimization in multiview 3D reconstruction. International Journal of Computer Vision 84(1), 80–96 (2009)
Lee, C.H., Varshney, A., Jacobs, D.W.: Mesh saliency. ACM Transactions on Graphics (TOG) 24, 659–666 (2005)
Lewis, J.: Fast normalized cross-correlation. Vision Interface 10, 120–123 (1995)
Liu, Y., Cao, X., Dai, Q., Xu, W.: Continuous depth estimation for multi-view stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2121–2128. IEEE (2009)
Meyer, M., Desbrun, M., Schröder, P., Barr, A.H.: Discrete differential-geometry operators for triangulated 2-manifolds. In: Visualization and Mathematics III, pp. 35–57. Springer (2003)
Nordström, M., Järvstråt, N.: An appearance-based measure of surface defects. International Journal of Material Forming 2(2), 83–91 (2009)
Oren, M., Nayar, S.K.: Generalization of lambert’s reflectance model. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, pp. 239–246. ACM (1994)
Reddy, J.N., Gartling, D.K.: The finite element method in heat transfer and fluid dynamics. CRC Press (2010)
Schroers, C., Zimmer, H., Valgaerts, L., Bruhn, A., Demetz, O., Weickert, J.: Anisotropic range image integration. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 73–82. Springer, Heidelberg (2012)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 519–528. IEEE (2006)
Stewenius, H., Schaffalitzky, F., Nister, D.: How hard is 3-view triangulation really? In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 686–693. IEEE (2005)
Sun, J., Li, Y., Kang, S.B., Shum, H.Y.: Symmetric stereo matching for occlusion handling. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 399–406. IEEE (2005)
Szeliski, R., Kang, S.B.: Shape ambiguities in structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(5), 506–512 (1997)
Turk, G., Levoy, M.: Zippered polygon meshes from range images. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, pp. 311–318. ACM (1994)
Wardetzky, M., Mathur, S., Kälberer, F., Grinspun, E.: Discrete laplace operators: no free lunch. In: Symposium on Geometry Processing, pp. 33–37 (2007)
Wright, S., Nocedal, J.: Numerical optimization, vol. 2. Springer, New York (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
1 Electronic Supplementary Material
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Semerjian, B. (2014). A New Variational Framework for Multiview Surface Reconstruction. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8694. Springer, Cham. https://doi.org/10.1007/978-3-319-10599-4_46
Download citation
DOI: https://doi.org/10.1007/978-3-319-10599-4_46
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10598-7
Online ISBN: 978-3-319-10599-4
eBook Packages: Computer ScienceComputer Science (R0)