Abstract
In 3DTV research, virtual view synthesis is a key component to the technology. Depth-image-based-rendering (DIBR) is an important method to realize virtual view synthesis. However, DIBR always results in hole problems where the depth and colour values are not known. Hole-filling methods often cause other problems, such as edge-ghosting and cracks. This paper proposes an algorithm that uses the depth and colour images to address the holes. It exploits the assumption of a virtual view between two laterally aligned reference cameras. The hole-filling method is performed on the blended depth image by morphological operations, and inpainting of the holes is obtained with the position information provided by the filtered depth maps. A new interpolation method to eliminate edge-ghosting is also presented, which additionally uses a post-processing technique to improve image quality. The main novelty of this paper is the unique image blending, which is more efficient than pre-processing depth maps. It is also the first method that is using morphological closing in the depth map de-noising process. The method proposed in this paper can effectively remove holes and edge-ghosting. Experimental quantitative and qualitative results show the proposed algorithm improves quality dramatically on traditional methods.
Chapter PDF
Similar content being viewed by others
References
Vazquez, C., Tam, W.J., Speranza, F.: Sterescopic Imaging: Filling Disoccluded Areas in Depth Image-Based Rendering. In: Proceedings of SPIE, Orlando, FL, USA, vol. 6392 (2006)
Domaski, M., Gotfryd, M., Wegner, K.: View Synthesis For Multiview Video Transmission. In: International Conference on Computer Vision and Pattern Recognition, Florida, USA, pp. 433–439 (2009)
Narayanan, P., Kumar, P., Reddy, K.: Depth+Texture Representation For Image Based Rendering. In: Proceedings of Fourth Indian Conference on Computer Vision, Graphics and Image Processing, Kolkata, Indian, pp. 113–118 (2004)
Fehn, C.: Depth-image-based rendering(DIBR), Compression, and Transmission For a New Approach on 3DTV. In: Proceedings of the SPIE, San Jose, CA, USA, vol. 5291, pp. 93–104 (2004)
McMillan, L.: An Image-Based Approach to Three-Dimensional Computer Graphics. Technical Report. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA (1997)
Mori, Y., Fukushima, N., Yendo, T., Fujii, T., Tanimoto, M.: View Generation With 3D Warping Using Depth Information for FTV. Signal Processing: Image Communication 24(1), 65–72 (2009)
Li, M., Chen, H., Li, R., Chang, X.: An Improved Virtual View Rendering Method Based on Depth Image. In: International Conference on Computer Communication, Jinan, China, pp. 381–384 (2011)
Zhang, L., Tam, J., Wang, D.: Stereoscopic Image Generation Based on Depth Images For 3DTV. IEEE Transactions on Broadcasting 51(2), 191–199 (2005)
Criminisi, A., Perez, P., Toyama, K.: Region Filling and Object Removal by Exemplar-Based Image Inpainting. IEEE Transactions on Image Processing 13(9), 1200–1212 (2004)
Oh, K., Yea, S., Vetro, A., Ho, Y.: Virtual View Synthesis Method and Self Evaluation Metrics for Free Viewpoint Television and 3D Video. International Journal of Imaging Systems and Technology 20(4), 378–390 (2010)
Jung, J., Ho, Y.: Virtual View Synthesis Using Temporal Hole Filling with Bilateral Coefficients. In: IEEE International Conference on Research, Innovation and Vision for the Future, pp. 1–4 (2012)
Herk, M.V.: A Fast Algorithm for Local Minimum and Maximum Filters on Rectangular and Octagonal Kernels. Patt. Recog. Letters 13, 517–521 (1992)
Canny, J.: A Computational Approach To Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8(6), 679–698 (1986)
Yu, Y., Acton, S.: Speckle Reducing Anisotropic Diffusion. IEEE Transactions on Image Processing 11, 1260–1270 (2002)
Microsoft Research, Image-Based Realities-3D Video Download, http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/.
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, Y., Chen, H., Gao, W., Vaudrey, T. (2014). Virtual View Synthesis Based on DIBR and Image Inpainting. In: Klette, R., Rivera, M., Satoh, S. (eds) Image and Video Technology. PSIVT 2013. Lecture Notes in Computer Science, vol 8333. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53842-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-53842-1_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-53841-4
Online ISBN: 978-3-642-53842-1
eBook Packages: Computer ScienceComputer Science (R0)