Abstract
Street-level visualization is an important application of 3D city models. Challenges to street-level visualization include the cluttering of buildings due to fine detail and visualization performance. In this paper, a novel method is proposed for street-level visualization based on visual saliency evaluation. The basic idea of the method is to preserve these salient buildings in a scene while removing those that are non-salient. The method can be divided into pre-processing procedures and real-time visualization. The first step in pre-processing is to convert 3D building models at higher Levels of Detail (LoDs) into LoD1 models with simplified ground plans. Then, a number of index viewpoints are created along the streets; these indices refer to both the position and the direction of each street site. A visual saliency value is computed for each building, with respect to the index site, based on a visual difference between the original model and the generalized model. We calculate and evaluate three methods for visual saliency: local difference, global difference and minimum projection area. The real-time visualization process begins by mapping the observer to its closest indices. The street view is then generated based on the building information stored in those indexes. A user study shows that the local visual saliency method performs better than do the global visual saliency, area and image-based methods and that the framework proposed in this paper may improve the performance of 3D visualization.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Anders K. 2005. Level of detail generation of 3D building groups by aggregation and typification. Proceedings of XXII International Cartographic Conference 2005, Spain
Anter K. 2000. What Colour is the Red House? Perceived colour of painted facades. Doctoral Dissertation. Stockholm: Royal Institute of Technology
Behr J, Eschler P, Jung Y, Zöllner M. 2009. X3DOM: A DOM-based HTML5/X3D integration model. Web3D’ 09, Darmstadt, Germany
Bruce N, Tsotsos J. 2005. Saliency Based on information maximization. Proc Neural Inf Process Syst (NIPS), 2005
International Commission on Illumination (CIE). 2007. CIE 1976 L*a*b* colour space draft standard. http://cie.co.at/index.php?i_ca_id=485. accessed Feb 13, 2014
Cole G G, Kentridge R W, Heywood C A. 2004. Visual saliency in the change detection paradigm: The special role of object onset. J Exp Psychol-Human Percept Performance, 30: 464–477
Elias B. 2003. Extracting landmarks with data mining methods. In: Kuhn W, Worboys M, Timpf S, eds. Spatial Information Theory: Foundations of Geographic Information Science. Lecture Notes Comput Sci, 2825: 375–389
Elias B, Hampe M, Monika S. 2005. Adaptive visualisation of landmarks using an MRDB. In: Meng L Q, Zipf A, Reichenbacher T, eds. Map-based Mobile Services-Theories, Methods and Implementations. Heidelberg: Springer. 73–86
Fan H, Meng L, Jahnke M. 2009. Generalization of 3D buildings modeled by CityGML. Lecture notes in geoinformation and cartography. Adv GISci, 387–405
Forberg A. 2007. Generalization of 3D building data based on a scale-space approach. ISPRS-J Photogramm Remote Sens, 62: 104–111
Guercke R, Götzelmann T, Brenner C, et al. 2011. Aggregation of LoD 1 building models as an optimization problem. ISPRS-J Photogramm Remote Sens, 66: 209–222
Hara K, Le V, Froehlich J. 2013. Combining crowdsourcing and google street view to identify street-level accessibility problems. In: Proc of the SIGCHI Conference on Human Factors in Computing Syst (CHI’ 13). New York: ACM. 631–640
Harel J. 2013. A Saliency Implementation in MATLAB: http://www.klab.caltech.edu/~harel/share/gbvs.php. accessed Nov 12 2013
Harel J, Koch C, Perona P. 2006. Graph-based visual saliency. In: Schölkopf B, Platt J C, Hoffman T, eds. Advances, Proceeding of Neural Information Process Systems 19 (NIPS 2006)
Hitchcock F L. 1941. The distribution of a product from several sources to numerous localities. J Math Phys, 20: 224–230
Hoffmann G. 2003. CIELAB Color Space, online at: http://www.fho-emden.de/~hoffmann/cielab03022003.pdf. accessed Nov 12, 2013
Itti L. 2004. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans Image Process, 13: 1304–1318
Itti L, Koch C. 2000. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res, 40: 1489–1506
Itti L, Koch C, Niebur E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell, 20: 1254–1259
JTS. 2013. http://www.vividsolutions.com/jts/JTSHome.htm. accessed Nov 12, 2013
Kada M. 2002. Automatic generalization of 3D building models. Int Arch Photogramm Remote Sens Spatial Inf Sci, 34: 243–248
Kim D H, Yun I D, Lee S U. 2004. A new attributed relational graph matching algorithm using the nested structure of earth mover’s distance. In: Proc of 17th Int Conf Pattern Recognition (ICPR’04)
Kim Y, Varshney A. 2006. Saliency-guided enhancement for volume visualization. IEEE Trans Vis Comput Graph 12: 925–932
Kolbe T H. 2008. Representing and Exchanging 3D City Models with CityGML in 3D Geo-Information Sciences. Berlin: Springer
Kopf J, Chen B, Szeliski R, et al. 2010. Street Slide: Browsing Street Level Imagery. ACM Trans Graph, 29:96: 1–8
Lee C H, Kim Y, Varshney A. 2009. Saliency-guided lighting. IEICE Trans Inf Systems, E92.D: 369–373
Lee T. 2009. Robust 3D street-view reconstruction using sky motion estimation. In: Proceedings of IEEE Int Workshop on 3-D Digital Imaging and Modeling (3DIM2009) in Conjunction with ICCV
Mao B, Fan H, Harrie L, et al. 2010. City model generalization similarity measurement using nested earth mover’s distance. In: Proceedings of 13th Workshop of the ICA commission on Generalisation and Multiple Representation, Zurich, Switzerland, 12–13 September
Mao B, Ban Y. 2011. Visualization of 3D city model through the internet using CityGML and X3DOM. Cartographica, 46: 109–114
Mao B, Harrie L, Ban Y. 2011. Detection and typification of linear structures for dynamic visualisation of 3D city models. Computs Environ Urban Syst, 36: 233–244
Mao B, Ban Y. 2013. Generalization of 3D building texture using image compression and multiple representation data structure. ISPRS-J Photogramm Remote Sens, 79: 68–79
Menzel N, Guthe M. 2010. Towards perceptual simplification of models with arbitrary materials. Comput Graphics Forum, 29: 2261–2270
Miao Y, Feng J. 2010. Perceptual-saliency extremum lines for 3D shape illustration. Visual Comput, 26: 433–443
Micusik B, Kosecka J. 2009. Piecewise planar city 3D modeling from street view panoramic sequences. IEEE Conf Comput Vision Pattern Recognition (CVPR), Miami, USA
Mortara M, Spagnuolo M. 2009. Semantics-driven best view of 3D shapes. Comput Graph, 33: 280–290
Nurminen A. 2008. Mobile 3D city maps. Comput Graphics, 28: 20–31
Parry M, Ribarsky W, Shaw C, et al. 2002. Organization and simplification of high-resolution 3D city facades. In: Proceedings of SPIE Aerosense Symposium, Visualization of Temporal and Spatial Data for Civilian and Defense Applications IV, 4744B
Rakkolainen I, Vainio T. 2001. A 3D city information for mobile users. Comput Graphics, 25: 619–625
Raubal M, Winter S. 2002. Enriching way finding instructions with local landmarks. Lecture Notes Comput Sci, 2478: 243–259
Salmen J, Houben S, Schlipsing M. 2012. Google street view images support the development of vision-based driver assistance systems. Intell Vehicles Symposium. 891–895
Schulte-Coerne T. 2005. Visualisierung von distanzfunktionen insbesondere der earth movers distance. GI Inf 2005, Schloss Birlinghoven: 8. bis 9. April 2005
Sester M, Brenner C. 2000. Typification based on Kohonen feature nets. The 1st International Conference on Geographic Information Science (GIScience 2000) Savannah, Georgia, USA
Sester M, Brenner C. 2004. Continuous generalization for fast and smooth visualization on small displays. Int Arch Photogramm Remote Sens, 34: 1293–1298
Frintrop S, Jensfelt P, Christensen H. 2006. Attentional landmark selection for visual SLAM. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’06), October 2006
Sun X, Qing W H, Hua Y W. 2009. Design and implementation of global 3D visualization client program on intelligent mobile devices. Third Int Conf Multimedia Ubiquitous Eng, 459–464
Thiemann F. 2002. Generalization of 3D building data. The Int Arch Photogramm Remote Sens Spatial Inf Sci, 34: 286–290
Thompson K G, Bichot N P. 2005. A visual saliency map in the primate frontal eye field. Prog Brain Res, 147: 251–262
Vincent L. 2007. Taking online maps down to street level. Compute, 40: 118–120
Wang M, Li J, Huang T, et al. 2010. Saliency detection based on 2D log-gabor wavelets and center bias. ACM Multimedia, 2010: 979–982
Winter S. 2003. Route adaptive selection of salient features. COSIT, 2003: 349–361
Xiao J, Fang T, Zhao P, et al. 2009. Image-based street-side city modeling. ACM Trans Graphics, 28: 114
Xj3D. 2013. http://www.xj3d.org/. accessed Nov 12, 2013
Yantis S. 2005. How visual salience wins the battle for awareness. Nat Neurosci 8: 975–977
Zhu Q, Li D, Zhang Y. 2002. Integration of DEMs, images and 3D models. Photogramm Eng Remote Sens, 68: 361–367
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mao, B., Ban, Y. & Harrie, L. Real-time visualization of 3D city models at street-level based on visual saliency. Sci. China Earth Sci. 58, 448–461 (2015). https://doi.org/10.1007/s11430-014-4955-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11430-014-4955-8