Abstract
We propose a variational algorithm to jointly estimate the shape, albedo, and light configuration of a Lambertian scene from a collection of images taken from different vantage points. Our work can be thought of as extending classical multi-view stereo to cases where point correspondence cannot be established, or extending classical shape from shading to the case of multiple views with unknown light sources. We show that a first naive formalization of this problem yields algorithms that are numerically unstable, no matter how close the initialization is to the true geometry. We then propose a computational scheme to overcome this problem, resulting in provably stable algorithms that converge to (local) minima of the cost functional. We develop a new model that explicitly enforces positivity in the light sources with the assumption that the object is Lambertian and its albedo is piecewise constant and show that the new model significantly improves the accuracy and robustness relative to existing approaches.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Abraham, R., Marsden, J. E., & Ratiu, T. (1993). Applied Mathematical Sciences: Vol. 75. Manifolds, tensor analysis, and applications (2nd ed.). New York: Springer.
Belhumeur, P., Kriegman, D., & Yuille, A. L. (1999). The generalized bas relief ambiguity. International Journal of Computer Vision, 35, 33–44.
Brox, T., & Weickert, J. (2006). Level set segmentation with multiple regions. IEEE Transactions on Image Processing, 15(10).
Chefd’hotel, C., Tschumperlé, D., Deriche, R., & Faugeras, O. (2004). Regularizing flows for constrained matrix-valued images. Journal of Mathematical Imaging and Vision, 20(1–2), 147–162.
Chen, H. F., Belhumeur, P. N., & Jacobs, D. W. (2000). In search of illumination invariants. In Proceedings of the IEEE conference on computer vision and pattern recognition
Durou, J.-D., Falcone, M., & Sagona, M. (2004). A survey of numerical methods for shape from shading. Research report 2004-2-R, IRIT, January 2004.
Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, pdes, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344.
Horn, B. K. P. (1986). Robot vision. Cambridge: MIT.
Horn, B., & Brooks, M. (Eds.). (1989). Shape from shading. Cambridge: MIT.
Jin, H. (2003). Variational methods for shape reconstruction in computer vision. PhD thesis, Electrical Engineering Department, Washington University, August 2003.
Jin, H., Soatto, S., & Yezzi, A. J. (2003a). Multi-view stereo beyond Lambert. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. I, pp. 171–178), June 2003.
Jin, H., Yezzi, A. J., Tsai, Y.-H., Cheng, L.-T., & Soatto, S. (2003b). Estimation of 3D surface shape and smooth radiance from 2D images: a level set approach. Journal of Scientific Computing, 19(1–3), 267–292.
Jin, H., Cremers, D., Yezzi, A., & Soatto, S. (2004a). Shedding light on stereoscopic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 36–42).
Jin, H., Yezzi, A. J., & Soatto, S. (2004b). Region-based segmentation on evolving surfaces with application to 3d shape and radiance estimation. In Proceedings of the European conference on computer vision (pp. 114–125), May 2004.
Jin, H., Yezzi, A., & Soatto, S. (2007, in press). Mumford-shah on the move: region-based segmentation on deforming manifolds with application to 3-D reconstruction of shape and appearance from multi-view images. Journal of Mathematical Imaging and Vision.
Keriven, R., Pons, J.-P., & Faugeras, O. (2006). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. To appear in the international journal of computer vision. International Journal of Computer Vision
Klette, R., Kozera, R., & Schlüns, K. (1998). Shape from shading and photometric stereo methods. Technical Report CITR-TR-20, University of Auckland, New Zealand.
Koenderink, J., & van Doorn, A. (1980). Photometric invariants related to solid shape. Optica Acta, 27(7), 981–996.
Kolev, K., Brox, T., & Cremers, D. (2006). Robust variational segmentation of 3D objects from multiple views. In K. Franke et al. (Eds.), Lecture notes in computer science: Vol. 4174. Pattern recognition (proceedings DAGM) (pp. 688–697). Berlin: Springer.
Kuhn, H. W., & Tucker, A. W. (1951). Nonlinear programming. In J. Neyman (Ed.), Proceedings of the second Berkeley symposium on mathematical statistics and probability (pp. 481–492) Berkeley: University of California Press.
Kutulakos, K. N., & Seitz, S. M. (2000). A theory of shape by space carving. International Journal of Computer Vision, 38(3), 199–218.
Langer, M. S., & Zucker, S. W. (1994). Shape from shading on a cloudy day. Journal of Optical Society of America, 11, 467–478.
Ma, Y., Soatto, S., Kosecka, J., & Sastry, S. (2003). An invitation to 3D vision, from images to models. Berlin: Springer.
Nayar, S., Ikeuchi, K., & Kanade, T. (1991). Surface reflection: physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 611–634.
Oliensis, J., & Dupuis, P. (1993). A global algorithm for shape from shading. In Proceedings of the international conference on computer vision (pp. 692–710).
Osher, S., & Sethian, J. (1988). Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi equations. Journal of Computational Physics, 79, 12–49.
Prados, E. (2004). Application of the theory of the viscosity solutions to the shape from shading problem. PhD thesis, Univ. of Nice-Sophia Antipolis.
Robles-Kelly, A., & Hancock, E. R. (2004). Estimating the surface radiance function from single images. In 3DPVT (pp. 494–501).
Robles-Kelly, A., & Hancock, E. R. (2005). Surface radiance correction for shape-from-shading. Pattern Recognition, 38(10), 1574–1595.
Samaras, D. (2003). Illumination constraints in deformable models for shape and light direction estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 247–264.
Samaras, D., Metaxas, D., Fua, P. V., & Leclerc, Y. G. (2000). Variable albedo surface reconstruction from stereo and shape from shading. In Proceedings of the IEEE international conference on computer vision and pattern recognition (Vol. I, pp. 480–487).
Seitz, S., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE international conference on computer vision and pattern recognition (pp. 519–526).
Stewart, A. J., & Langer, M. S. (1997). Towards accurate recovery of shape from shading under diffuse lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9), 1020–1025.
Urakawa, H. (1993). Calculus of variations and harmonic maps. Providence: American Mathematical Society.
Vedaldi, A., & Soatto, S. (2006). Viewpoint invariance for non-planar scenes. Technical Report TR050012, UCLA CSD.
Vese, L. A., & Chan, T. F. (2002). A multiphase level set framework for image processing using the Mumford–Shah functional. International Journal of Computer Vision, 50(3), 271–293.
Yezzi, A., & Soatto, S. (2001). Stereoscopic segmentation. In Proceedings of the international conference on computer vision (pp. 59–66).
Yuille, A. L., Snow, D., Epstein, R., & Belhumeur, P. (1999). Determining generative models of objects under varying illumination: shape and albedo from multiple images using svd and integrability. International Journal of Computer Vision, 35, 203–222.
Yuille, A., Coughlan, J. M., & Konishi, S. (2003). Kgbr viewpoint-lighting ambiguity. Journal of the Optical Society of America A, 20(1), 24–31.
Zhang, R., Tsai, P.-S., Cryer, J., & Shah, M. (1999). Shape from shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706.
Zheng, Q., & Chellappa, R. (1991). Estimation of illuminant direction, albedo and shape from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 680–702.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jin, H., Cremers, D., Wang, D. et al. 3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination. Int J Comput Vis 76, 245–256 (2008). https://doi.org/10.1007/s11263-007-0055-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0055-y