Abstract
This paper presents a volumetric stereo and silhouette fusion algorithm for acquiring high quality models from multiple calibrated photographs. Our method is based on computing and merging depth maps. Different from previous methods of this category, the silhouette information is also applied in our algorithm to recover the shape information on the textureless and occluded areas. The proposed algorithm starts by computing visual hull using a volumetric method in which a novel projection test method is proposed for visual hull octree construction. Then, the depth map of each image is estimated by an expansion-based approach that returns a 3D point cloud with outliers and redundant information. After generating an oriented point cloud from stereo by rejecting outlier, reducing scale, and estimating surface normal for the depth maps, another oriented point cloud from silhouette is added by carving the visual hull octree structure using the point cloud from stereo to restore the textureless and occluded surfaces. Finally, Poisson Surface Reconstruction approach is applied to convert the oriented point cloud both from stereo and silhouette into a complete and accurate triangulated mesh model. The proposed approach has been implemented and the performance of the approach is demonstrated on several real data sets, along with qualitative comparisons with the state-of-the-art image-based modeling techniques according to the Middlebury benchmark.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Prados, E., Faugeras, O.: Perspective shape from shading and viscosity solutions. In: ICCV 2, pp. 826–831 (2003)
Szeliski, R.: Rapid octree construction from range sequences. Comput. Vis. Graph. Image Process. 58(1), 23–32 (1993)
Yemez, Y., Schmitt, F.: 3D reconstruction of real objects with high resolution shape and texture. Image Vis. Comput. 22(13), 1137–1153 (2004)
Kazhdan, M., Bolithp, M., Hoppe, H.: Poisson surface reconstruction. In: Eurographics Symposium on Geometry Processing, pp. 61–70 (2006)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: CVPR 1, pp. 519–526 (2006)
Vogiatzis, G., Torr, P., Cipolla, R.: Multi-view stereo via volumetric graph-cuts. In: CVPR, pp. 391–398 (2005)
Sinha, S.N., Mordohai, P., Pollefeys, M.: Multi-view stereo via graph cuts on the dual of an adaptive tetrahedral mesh. In: ICCV (2007)
Vogiatzis, G., Hernandez, C., Torr, P.H.S., Cipolla, R.: Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistenc. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2241–2246 (2007)
Kutulakos, K., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000)
Hernandez, C., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Comput. Vis. Image Underst. 96(3), 367–392 (2004)
Pons, J., Keriven, R., Faugeras, O.: Modelling dynamic scenes by registering multi-view image sequences. In: CVPR, pp. 822–827 (2005)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: CVPR (2007)
Habbecke, M., Kobbelt, L.: A surface-growing approach to multi-view stereo reconstruction. In: CVPR (2007)
Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV (2007)
Narayanan, P., Rander, P., Kanade, T.: Constructing virtual worlds using dense stereo. In: ICCV, pp. 3–10 (1998)
Goesele, M., Curless, B., Seitz, S.M.: Multi-view stereo revisited. In: CVPR, pp. 2402–2409 (2006)
Bradley, D., Boubekeur, T., Heidrich, W.: Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In: CVPR (2008)
Liu, Y., Cao, X., Dai, Q., Xu, W.: Continuous depth estimation for multi-view stereo. In: CVPR, pp. 2121–2128 (2009)
Laurentini, A.: The visual hull concept for silhouette based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994)
Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: Proc. 12th Eurographics Workshop on Rendering, pp. 115–125 (2001)
Tarini, M., Callieri, M., Montani, C., Rocchini, C., Olsson, K., Persson, T.: Marching intersections: An efficient approach to shape-from-silhouette. In: Proceedings of VMV, pp. 283–290 (2002)
Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3d surface construction algorithm. In: SIGGRAPH 21, pp. 163–169 (1987)
Song, P., Wu, X., Wang, M.Y.: A robust and accurate method for visual hull computation. In: IEEE International Conference on Information and Automation, pp. 784–789 (2009)
Potmesil, M.: Generating octree models of 3d objects from their silhouettes in a sequence of images. Comput. Vis. Graph. Image Process. 40, 1–29 (1987)
Borgefors, G.: Distance transformations in digital images. Comput. Vis. Graph. Image Process. 34, 344–371 (1986)
Jarvis, R.A.: On the identification of the convex hull of a finite set of points in the plane. Inf. Process. Lett. 2, 18–21 (1973)
Hernandez, C., Schmitt, F.: Multi-stereo 3d object reconstruction. In: 3DPVT, pp. 159–166 (2002)
Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., Stuetzle, W.: Surface reconstruction from unorganized points. In: SIGGRAPH, pp. 71–78 (1992)
Yemez, Y., Wetherilt, C.J.: A volumetric fusion technique for surface reconstruction from silhouette and range data. Comput. Vis. Image Underst. 105(1), 30–41 (2007)
Poisson Surface Reconstruction package. http://www.cs.jhu.edu/misha/Code/PoissonRecon/
BigHead sequence. http://www.tsi.enst.fr/3dmodels/
3D photography data sets. http://www-cvr.ai.uiuc.edu/yfurukaw/research/mview
Dino and Temple data sets. http://vision.middlebury.edu/mview/
Furukawa, Y., Ponce, J.: Carved visual hulls for imaged-based modeling. In: ECCV, vol. 1, pp. 564–577 (2006)
Vu, H., Keriven, R., Labatut, P., Pons, J.P.: Towards high-resolution large-scale multi-view stereo. In: CVPR (2009)
Lhuillier, M., Quan, L.: Match propagation for image-based modeling and rendering. IEEE Trans. Pattern. Anal. Mach. Intell. 24(8), 1140–1146 (2002)
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Song, P., Wu, X. & Wang, M.Y. Volumetric stereo and silhouette fusion for image-based modeling. Vis Comput 26, 1435–1450 (2010). https://doi.org/10.1007/s00371-010-0429-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-010-0429-y