Abstract
This paper presents a method for detecting a textured deformed surface in an image. It uses (wide-baseline) point matches between a template and the input image. The main contribution of the paper is twofold. First, we propose a robust method based on local surface smoothness capable of discarding outliers from the set of point matches. Our method handles large proportions of outliers (beyond 70% with less than 15% of false positives) even when the surface self-occludes. Second, we propose a method to estimate a self-occlusion resistant warp from point matches. Our method allows us to realistically retexture the input image. A pixel-based (direct) registration approach is also proposed. Bootstrapped by our robust point-based method, it finely tunes the warp parameters using the value (intensity or color) of all the visible surface pixels. The proposed framework was tested with simulated and real data. Convincing results are shown for the detection and retexturing of deformed surfaces in challenging images.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: a unifying framework. International Journal of Computer Vision, 56(3), 221–255.
Bartoli, A. (2008). Maximizing the predictivity of smooth deformable image warps through cross-validation. Journal of Mathematical Imaging and Vision, 31(2), 133–145.
Bartoli, A., & Zisserman, A. (2004). Direct estimation of non-rigid registrations. In British machine vision conference (vol. 2, pp. 899–908).
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008). Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3), 346–359.
Bookstein, F. L. (1989). Principal warps: thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(6), 567–585.
Chui, H., & Rangarajan, A. (2003). A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding, 89(2–3), 114–141.
DeCarlo, D., & Metaxas, D. (1998). Deformable model-based shape and motion analysis from images using motion residual error. In International conference on computer vision (vol. 39, pp. 380–393).
Duchon, J. (1976). Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Analyse Numérique, 10, 5–12.
Gay-Bellile, V., Bartoli, A., & Sayd, P. (2010). Direct estimation of nonrigid registrations with image-based self-occlusion reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 87–104.
Hilsmann, A., & Eisert, P. (2009). Tracking and retexturing cloth for real-time virtual clothing applications. In Computer vision/computer graphics collaboration techniques: 4th international conference, MIRAGE 2009, Rocquencourt, France, May 4–6, 2009 (pp. 94–105). New York: Springer.
Lepetit, V., & Fua, P. (2006). Keypoint recognition using randomized trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9), 1465–1479.
Lischinski, D. (1994). Incremental delaunay triangulation. Graphics Gems, IV, 47–59.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.
Pilet, J., Lepetit, V., & Fua, P. (2008). Fast non-rigid surface detection, registration and realistic augmentation. International Journal of Computer Vision, 76(2), 109–122.
Pizarro, D., & Bartoli, A. (2010). Feature-based deformable surface detection with self-occlusion reasoning. In Electronic proceedings of the fifth international symposium on 3D data processing, visualization and transmission, 3DPVT’10.
Pizarro, D., Peyras, J., & Bartoli, A. (2008). Light-invariant fitting of active appearance models. In Proc. IEEE conf. on computer vision and pattern recognition, CVPR 2008, Anchorage, Alaska, USA (pp. 1–6).
Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999). Nonrigid registation using free-form deformations: application to breast MR images. IEEE Transactions on Medical Imaging, 18(8), 712–721.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pizarro, D., Bartoli, A. Feature-Based Deformable Surface Detection with Self-Occlusion Reasoning. Int J Comput Vis 97, 54–70 (2012). https://doi.org/10.1007/s11263-011-0452-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-011-0452-0