Abstract
Shape-from-Shading (SfS) is one of the fundamental techniques to recover depth from a single view. Such a method has shown encouraging but limited results in laparoscopic surgery due to the complex reflectance properties of the organ tissues. On the other hand, Template-Based Deformable-Shape-from-Motion (DSfM) has been recently used to recover a coarse 3D shape in laparoscopy.
We propose to combine both geometric and photometric cues to robustly reconstruct 3D human organs. Our method is dubbed Deformable-Shape-from-Motion-and-Shading (DSfMS). It tackles the limits of classical SfS and DSfM methods: First the photometric template is reconstructed using rigid SfM (Shape-from-Motion) while the surgeon is exploring – but not deforming – the peritoneal environment. Second a rough 3D deformed shape is computed using a recent method for elastic surface from a single laparoscopic image. Third a fine 3D deformed shape is recovered using shading and specularities.
The proposed approach has been validated on both synthetic data and in-vivo laparoscopic videos of a uterus. Experimental results illustrate its effectiveness compared to SfS and DSfM.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Brunet, F., Hartley, R., Bartoli, A., Navab, N., Malgouyres, R.: Monocular Template-Based Reconstruction of Smooth and Inextensible Surfaces. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 52–66. Springer, Heidelberg (2011)
Grasa, O.G., Civera, J., Montiel, J.: EKF monocular slam with relocalization for laparoscopic sequences. In: ICRA (2011)
Hager, G., Vagvolgyi, B., Yuh, D.: Stereoscopic video overlay with deformable registration. Medicine Meets Virtual Reality (2007)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2003)
Hayashibe, M., Suzuki, N., Hattori, A., Nakamura, Y.: Intraoperative Fast 3D Shape Recovery of Abdominal Organs in Laparoscopy. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2489, pp. 356–363. Springer, Heidelberg (2002)
Hayashibe, M., Suzuki, N., Nakamura, Y.: Laser-scan endoscope system for intraoperativegeometry acquisition and surgical robot safety management. MIA 10, 509–519 (2006)
Malti, A., Bartoli, A., Collins, T.: Template-based conformal shape-from-motion from registered laparoscopic images. In: MIUA (2011)
Moreno-Noguer, F., Salzmann, M., Lepetit, V., Fua, P.: Capturing 3D stretchable surfaces from single images in closed form. In: CVPR (2009)
Morris, D.D., Kanade, T.: Image-consistent surface triangulation. In: CVPR (2000)
Nicolau, S.A., Pennec, X., Soler, L., Ayache, N.: A Complete Augmented Reality Guidance System for Liver Punctures: First Clinical Evaluation. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3749, pp. 539–547. Springer, Heidelberg (2005)
Nishino, K., Zhang, Z., Ikeuchi, K.: Illumination distribution from a sparse set of images for view-dependent image synthesis. In: ICCV (2001)
Penne, J., Höller, K., Stürmer, M., Schrauder, T., Schneider, A., Engelbrecht, R., Feußner, H., Schmauss, B., Hornegger, J.: Time-of-Flight 3-D Endoscopy. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5761, pp. 467–474. Springer, Heidelberg (2009)
Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. In: BMVC (2008)
Prados, E., Camilli, F., Faugeras, O.: A unifying and rigorous shape from shading method adapted to realistic data and applications. Journal of Mathematical Imaging and Vision 25(3), 307–328 (2006)
Salzmann, M., Urtasun, R., Fua, P.: Local deformation models for monocular 3D shape recovery. In: CVPR (2008)
Stoyanov, D., Darzi, A., Yang, G.Z.: Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 41–48. Springer, Heidelberg (2004)
Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery. In: Jiang, T., et al. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)
Totz, J., Mountney, P., Stoyanov, D., Yang, G.-Z.: Dense Surface Reconstruction for Enhanced Navigation in MIS. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011, Part I. LNCS, vol. 6891, pp. 89–96. Springer, Heidelberg (2011)
Winter, M.: Image-Based Incremental Reconstruction, Rendering and Augmented Visualziation of Surfaces for Endoscopic Surgery. PhD thesis, University of Erlangen (2009)
Wu, C., Varanasi, K., Liu, Y., Seidel, H.-P., Theobalt, C.: Shading-based dynamic shape refinement from multi-view video under general illumination. In: ICCV (2011)
Yoo, D.: Three-dimensional morphing of similar shapes using a template mesh. International Journal of Precision Engineering and Manufacturing (2009)
Zhang, L., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In: ICCV (2003)
Zhang, R., Tsai, P.-S., Cryer, J.E., Shah, M.: Shape from shading: A survey. PAMI 21(8), 690–706 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Malti, A., Bartoli, A., Collins, T. (2012). Template-Based Conformal Shape-from-Motion-and-Shading for Laparoscopy. In: Abolmaesumi, P., Joskowicz, L., Navab, N., Jannin, P. (eds) Information Processing in Computer-Assisted Interventions. IPCAI 2012. Lecture Notes in Computer Science, vol 7330. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30618-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-30618-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30617-4
Online ISBN: 978-3-642-30618-1
eBook Packages: Computer ScienceComputer Science (R0)