Abstract
Recovery of three dimensional (3D) shape and motion of non-static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction. If every point in the scene randomly moves, it is impossible to recover the non-rigid shapes. In practice, many non-rigid objects, e.g. the human face under various expressions, deform with certain structures. Their shapes can be regarded as a weighted combination of certain shape bases. Shape and motion recovery under such situations has attracted much interest. Previous work on this problem [6,4,13] utilized only orthonormality constraints on the camera rotations (rotation constraints). This paper proves that using only the rotation constraints results in ambiguous and invalid solutions. The ambiguity arises from the fact that the shape bases are not unique because their linear transformation is a new set of eligible bases. To eliminate the ambiguity, we propose a set of novel constraints, basis constraints, which uniquely determine the shape bases. We prove that, under the weak-perspective projection model, enforcing both the basis and the rotation constraints leads to a closed-form solution to the problem of non-rigid shape and motion recovery. The accuracy and robustness of our closed-form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Baker, S., Matthews, I.: Equivalence and Efficiency of Image Alignment Algorithms. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)
Bascle, B., Blake, A.: Separability of Pose and Expression in Facial Tracing and Animation. In: Proc. Int. Conf. Computer Vision, pp. 323–328 (1998)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proc. SIGGRAPH 1999, pp. 187–194 (1999)
Brand, M.: Morphable 3D Models from Video. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)
Brand, M., Bhotika, R.: Flexible Flow for 3D Nonrigid Tracking and Shape Recovery. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering Non-Rigid 3D Shape from Image Streams. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2000)
Chai, J., Xiao, J., Hodgins, J.: Vision-based Control of 3D Facial Animation. In: Eurographics/ACM Symposium on Computer Animation (2003)
Costeira, J., Kanade, T.: A multibody factorization method for independently moving-objects. Int. Journal of Computer Vision 29(3), 159–179 (1998)
Gokturk, S.B., Bouguet, J.Y., Grzeszczuk, R.: A data driven model for monocular face tracking. In: Proc. Int. Conf. Computer Vision (2001)
Han, M., Kanade, T.: Reconstruction of a Scene with Multiple Linearly Moving Objects. Proc. Int. Conf. Computer Vision and Pattern Recognition (2000)
Poelman, C., Kanade, T.: A paraperspective factorization method for shape and motion recovery. IEEE Trans. Pattern Analysis and Machine Intelligence 19(3), 206–218 (1997)
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. Int. Journal of Computer Vision 9(2), 137–154 (1992)
Torresani, L., Yang, D., Alexander, G., Bregler, C.: Tracking and Modeling Non- Rigid Objects with Rank Constraints. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)
Triggs, B.: Factorization Methods for Projective Structure and Motion. Proc. Int. Conf. Computer Vision and Pattern Recognition (1996)
Vidal, R., Soatto, S., Ma, Y., Sastry, S.: Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes (2002)
Wolf, L., Shashua, A.: Two-body Segmentation from Two Perspective Views. In: Proc. Int. Conf. Computer Vision and Pattern Recognition (2001)
Wolf, L., Shashua, A.: On Projection Matrices Pk → P2, k = 3,., 6, and their Applications in Computer Vision. Int. Journal of Computer Vision 48(1), 53–67 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xiao, J., Chai, Jx., Kanade, T. (2004). A Closed-Form Solution to Non-rigid Shape and Motion Recovery. In: Pajdla, T., Matas, J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, vol 3024. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24673-2_46
Download citation
DOI: https://doi.org/10.1007/978-3-540-24673-2_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21981-1
Online ISBN: 978-3-540-24673-2
eBook Packages: Springer Book Archive