Abstract
We consider a face-to-face videoconferencing system that uses a Kinect camera at each end of the link for 3D modeling and an ordinary 2D display for output. The Kinect camera allows a 3D model of each participant to be transmitted; the (assumed static) background is sent separately. Furthermore, the Kinect tracks the receiver’s head, allowing our system to render a view of the sender depending on the receiver’s viewpoint. The resulting motion parallax gives the receivers a strong impression of 3D viewing as they move, yet the system only needs an ordinary 2D display. This is cheaper than a full 3D system, and avoids disadvantages such as the need to wear shutter glasses, VR headsets, or to sit in a particular position required by an autostereo display. Perceptual studies show that users experience a greater sensation of depth with our system compared to a typical 2D videoconferencing system.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Rosenthal, A. H. Two-way television communication unit. US Patent 2420198, 1947
Okada, K.-I.; Maeda, F.; Ichikawaa, Y.; Matsushita, Y. Multiparty videoconferencing at virtual social distance: MAJIC design. In: Proceedings of ACM Conference on Computer Supported Cooperative Work, 385–393, 1994
Sellen, A.; Buxton, B.; Arnott, J. Using spatial cues to improve videoconferencing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 651–652, 1992
Tang, J. C.; Minneman, S. Video Whiteboard: Video shadows to support remote collaboration. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 315–322, 1991
Vertegaal, R. The GAZE groupware system: Mediating joint attention in multiparty communication and collaboration. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 294–301, 1999
Vertegaal, R.; Ding, Y. Explaining effects of eye gaze on mediated group conversations: Amount or synchronization? In: Proceedings of ACM Conference on Computer Supported Cooperative Work, 41–48, 2002
Pirenne, M. H. Optics, Painting and Photography. Cambridge, UK: Cambridge University Press, 1970
Solso, R. L. Cognition and the Visual Arts. Cambridge, MA, USA: MIT Press, 1996
Pepperell, R.; Haertel, M. Do artists use linear perspective to depict visual space? Perception Vol. 43, No. 5, 395–416, 2014
Baldwin, J.; Burleigh, A.; Pepperell, R. Comparing artistic and geometrical perspective depictions of space in the visual field. i-Perception Vol. 5, No. 6, 536–547, 2014
Kemp, M. The Science of Art: Optical Themes in Western Art from Brunelleschi to Seurat. New Haven, CT, USA: Yale University Press, 1990
Kingslake, R. Optics in Photography. Bellingham, WA, USA: SPIE Publications, 1992
Ogle, K. N. Research in Binocular Vision, 2nd edn. New York: Hafner Publishing Company, 1964
Harrison, C.; Hudson, S. E. Pseudo-3D video conferencing with a generic webcam. In: Proceedings of the 10th IEEE International Symposium on Multimedia, 236–241, 2008
Zhang, C.; Yin, Z.; Florencio, D. Improving depth perception with motion parallax and its application in teleconferencing. In: Proceedings of IEEE International Workshop on Multimedia Signal Processing, 1–6, 2009
Izadi, S.; Kim, D.; Hilliges, O.; Molyneaux, D.; Newcombe, R.; Kohli, P.; Shotton, J.; Hodges, S.; Freeman, D.; Davison, A.; Fitzgibbon, A. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 559–568, 2011
Newcombe, R. A.; Izadi, S.; Hilliges, O.; Molyneaux, D.; Kim, D.; Davison, A. J.; Kohli, P.; Shotton, J.; Hodges, S.; Fitzgibbon, A. KinectFusion: Real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, 127–136, 2011
Kim, K.; Bolton, J.; Girouard, A.; Cooperstock, J.; Vertegaal, R. TeleHuman: Effects of 3D perspective on gaze and pose estimation with a life-size cylindrical telepresence pod. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2531–2540, 2012
Lee, J. C. Head tracking for desktop VR displays using the Wii remote. Available at http://johnnylee.net/ projects/wii/.
iPhone User Guide For iOS 8.1 Software. Apple Inc., 2014
Levin, A.; Lischinski, D.; Weiss, Y. A closed form solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 2, 228–242, 2008
Rydfalk, M. CANDIDE, a parameterized face. Technical Report LiTH-ISY-I-866. Link¨oping University, 1987.
Welsh, B. Model-based coding of images. Ph.D. Thesis. British Telecom Research Lab, 1991
Ahlberg, J. CANDIDE-3—An updated parameterised face. Technical Report LiTH-ISY-R-2326. Linköping University, 2001
Rusinkiewicz, S.; Hall-Holt, O.; Levoy, M. Real-time 3D model acquisition. ACM Transactions on Graphics Vol. 21, No. 3, 438–446, 2002
3dMD Static Systems. Available at http://www.3dmd. com/3dMD-systems/.
Chen, J.; Bautembach, D.; Izadi, S. Scalable real-time volumetric surface reconstruction. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 113, 2013
Wexler, Y.; Shechtman, E.; Irani, M. Space–time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 3, 463–476, 2007
Chen, T.; Zhu, Z.; Shamir, A.; Hu, S.-M.; Cohen-Or, D. 3-Sweep: Extracting editable objects from a single photo. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 195, 2013
Gal, R.; Sorkine, O.; Mitra, N. J.; Cohen-Or, D. iWIRES: An analyze-and-edit approach to shape manipulation. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 33, 2009.
Schulz, A.; Shamir, A.; Levin, D. I. W.; Sitthi-amorn, P.; Matusik, W. Design and fabrication by example. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 62, 2014.
Zheng, Y.; Fu, H.; Cohen-Or, D.; Au, O. K.-C.; Tai, C.-L. Component-wise controllers for structurepreserving shape manipulation. Computer Graphics Forum Vol. 30, No. 2, 563–572, 2011
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Zhe Zhu is a Ph.D. candidate in the Department of Computer Science and Technology, Tsinghua University. He received his bachelor degree from Wuhan University in 2011. His research interests are computer vision and computer graphics.
Ralph R. Martin is currently a professor at Cardiff University. He obtained his Ph.D. degree in 1983 from Cambridge University. He has published more than 250 papers and 14 books, covering such topics as solid and surface modeling, intelligent sketch input, geometric reasoning, reverse engineering, and various aspects of computer graphics. He is a Fellow of the Learned Society of Wales, the Institute of Mathematics and its Applications, and the British Computer Society. He is on the editorial boards of Computer-Aided Design, Computer Aided Geometric Design, Geometric Models, the International Journal of Shape Modeling, CAD and Applications, and the International Journal of CADCAM. He was recently awarded a Friendship Award, China’s highest honor for foreigners.
Robert Pepperell is an artist who studied at the Slade School of Art, London, and has exhibited widely. He has published several books and numerous academic papers, and is a professor of fine art at Cardiff School of Art & Design in the UK. He specialises in research that combines art practice with scientific experimentation and philosophical inquiry.
Alistair Burleigh has a background in the conception and development of new creative digital ideas and technology for commercial application. He studied fine art: interactive media at Newport School of Art and went on to work in lead roles on creative digital projects for a wide range of functions and prestige clients on a global basis. He is now a researcher and technical director working at Cardiff School of Art & Design, UK.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Zhu, Z., Martin, R.R., Pepperell, R. et al. 3D modeling and motion parallax for improved videoconferencing. Comp. Visual Media 2, 131–142 (2016). https://doi.org/10.1007/s41095-016-0038-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-016-0038-4