Keywords

1 Introduction

Visual features play a crucial role in capturing and describing the distinctive characteristics of urban built environments. These visual features serve as vital input for various applications, including photogrammetry, 3D reconstruction, navigation, object recognition, object tracking and urban augmented reality [1, 2]. The significance of adequately describing visual features was recognized as early as the 1920s within the field of visual perception, leading to the establishment of fundamental concepts that have since influenced numerous approaches for feature extraction [2]. The rapid growth of urbanization and the increasing need for urban planning and analysis [3] have driven the development of advanced 3D geosimulation techniques [4,5,6]. Accurate modeling and representation of the urban built environment are crucial for realistic simulations and effective decision-making processes. In recent years, Pixar’s Universal Scene Description (USD) standard specifications have emerged as a powerful framework for modeling and managing complex 3D scenes, offering significant advantages in terms of interoperability, scalability, and extensibility [7, 8], which is appropriate for modern approaches in urban prospective simulation and analysis.

Combined with the third version of CityGML (City Geography Markup Language) in development [9, 10], the use of USD in the context of urban visual features modeling brings several advantages. First, the hierarchical and layer-based structure of USD allows for the modular composition and referencing of different visual elements, providing flexibility and ease of scene organization. This enables efficient updates and modifications to the models, enhancing the workflow and facilitating collaboration among researchers and stakeholders. Second, USD supports a wide range of data formats and can integrate diverse data sources, including aerial imagery, LiDAR data, GIS layers, and simulation outputs. This integration allows for a comprehensive representation of the urban built environment, incorporating both geometric and semantic information. The ability to merge and manage such heterogeneous data sources within a unified framework greatly enhances the realism and accuracy of the 3D geosimulation for different urban prospective and analysis. Third, USD’s robust schema and specification enable the precise definition of visual features, such as material properties, texture mapping, and lighting models. The physically-based rendering [11, 12] capabilities of USD ensure accurate and realistic rendering of urban elements, enhancing the visual quality and realism of the geosimulation results.

To the question “How to reconcile 3D geometry, time and the different semantics of spatial objects/agents for a truer representation of reality?”, the aim of this paper is to present a comprehensive approach to modeling visual features in the urban built environment using USD standard specifications.

After surveying keys existing standards and approaches for representing urban environments, we identified the challenges faced by digital twins and geosimulation and introduce the USD standard as a suitable solution in Sect. 2. Section 3 discusses the fundamentals and requirements of USD for urban 3D modeling. In Sect. 4, a series of experiments are presented, including experimental settings and result analysis, to showcase the advantages of the proposed USD approach compared to current methods and standards. The paper concludes in Sect. 5, summarizing the results and outlining future research directions.

2 Urban 3D Modeling and Motivation of USD Advocacy

2.1 Background and Related Works

The field of 3D data collection, storage, and management has matured, leading to increased utilization of 3D data [13, 14]. However, in urban contexts, there is a need for a comprehensive review to compare different 3D modeling methods and standards. This review would assess the efficacy, efficiency, and suitability of various approaches, provide valuable insights into advancements, and address evolving challenges. It would also guide future research and facilitate the adoption of standardized and effective 3D modeling techniques in urban domains. Contemporary 3D modeling approaches can be categorized into topological and geometric methods.

Topological modeling methods focus on preserving relationships between geometries, while geometric modeling methods directly capture geographical coordinates [15]. Integration of multiple modeling methods has gained popularity to address limitations and enhance efficiencies. Examples include combining B-rep and CSG (Constructive Solid Geometry) techniques [16, 17], as well as BIM (Building Information Modeling) and CityGML approaches, resulting in improved outcomes [18]. Integration of diverse techniques overcomes limitations, enhances accuracy, facilitates interoperability, and optimizes efficiencies [19,20,21]. The combination of methods represents a promising avenue for advancing 3D modeling and comprehensive representation of urban environments [22]. CityGML, developed by the Open Geospatial Consortium (OGC) [9], is widely adopted for comprehensive 3D representations [23]. The Unified Modeling Language (UML) is commonly used to define relationships in CityGML. Zlatanova et al. [24] conducted a comprehensive review of topological modeling methods in the context of urban environments.

Geometric modeling methods provide fast and efficient operations through direct access to object locations based on coordinates. However, they lack the ability to maintain adjacent topological relations, leading to data consistency issues. Geometric modeling methods can be categorized into points cloud, wireframe, mesh and voxel approaches. Points cloud modeling uses unstructured sets of points obtained from LiDAR data to create high-resolution and accurate 3D representations [11, 17, 24,25,26]. Wireframe modeling connects nodes to define the outer shapes of 3D objects [11], often utilizing point cloud data as input [17]. Mesh modeling, also known as 2.5D modeling, uses 2D data with height information to create 3D models. Procedural modeling methods, such as rule-based modeling, extrude 3D blocks based on 2D geoinformation and apply rules/algorithms for texture and facades [27,28,29,30,31]. Voxel and boundary representation (B-rep) employ vertices, edges, and faces to define geometric components. Voxel modeling utilizes regularly shaped grid points in 3D space, offering flexibility in adjusting scale for representation precision [16, 32]. BIM [25, 31] is parametric and integrates with 3D databases, following the Industry Foundation Classes (IFC) standard [33], for building-related information. BIM employs an object-oriented and standardized data definition language, like CityGML [34]. These various geometric modeling methods have their strengths and applications in 3D modeling, with BIM and CityGML representing significant advancements in the field [35].

2.2 From 3D City Modeling issues to Scientific Advocacy for USD

There is a wide array of geospatial software and tools available to support 3D data models and perform various functions, including viewing, generating, editing, converting, storing, parsing, and providing API (Application Programming Interface) for programmers. These software and tools are extensively utilized in certain exchange formats categorized as organizational standards such as CityGML, CityJSON, and IFC, while their usage is partial or limited in de facto standards like KML, SHP, DXF, COLLADA, and 3D PDF. Consequently, the presence of an open standardized data format holds crucial importance in the context of 3D models for geosimulation [14, 15, 20,21,22]. However, when it comes to developing effective visual analytics systems for 3D geosimulation, these existing standards and tools quickly reveal their limitations (Table 1).

Table 1. A comparative view of usual international 3D format standards.
Full size table

Although 3D modeling has significant potential for spatial analysis in complex urban areas, it remains a time-consuming and labor-intensive process. The existence of an internationally accepted data standard could address format harmonization issues [35,36,37,38,39]. Furthermore, there is a need for abundant fine-scale data, especially in areas like ventilation animation and emergency management [3, 10, 11]. Researchers in urban visual analytics must address questions regarding suitable visualization techniques, computational methods, and effective integration of visualization and computational models. Without answers to these questions, designing visual analytics solutions for urban problems becomes challenging. USD has been designed to overcome these challenges and enhance urban visual analytics by providing capabilities documented in Table 2. Leveraging USD can lead to more efficient and effective solutions in urban visual analytics.

Table 2. Challenges related to the limitations of current urban visual simulation standards.
Full size table

3 USD Fundamentals for Urban 3D Visual Features Modeling

3.1 Concepts and Requirements of Universal Scene Description Standard

In the context of modeling urban built environment visual features for 3D geosimulation, USD standard schema presents a robust and adaptable framework for representing 3D scenes and assets. Its layered composition and referencing capabilities facilitate the organization and management of intricate urban scenes. The schema provides a flexible and scalable framework for describing the geometry, attributes, relationships, and behaviors of objects within a scene. Employing a combination of JSON and binary formats, the schema ensures efficient storage and transmission of 3D data. By encompassing geometry, attributes, and metadata, it enables precise and detailed descriptions of visual features in files. Essentially, USD files contain data that dictates the appearance of a scene, which rendering applications interpret to generate images on the screen. There are several types of USD files. Readable ASCII text files have the.usda extension, offering human-readable representations. For more compact and efficient binary representations, the.usdc extension format is utilized. Additionally, USD supports a packaging format, denoted by the.usdz extension, which combines multiple USD files and associated auxiliary files (e.g., textures) within an uncompressed zip archive. The understanding and utilization of USD in urban visual modeling rely on key concepts. Geometry representation involves primitives like polygons, NURBS, curves, and points, enabling accurate depictions of urban elements. Instancing supports efficient rendering of repetitive objects. A stage serves as a hierarchical structure that organizes graphical information, comprising layers containing scene elements. Prims, the primary container objects, establish a hierarchy within the stage. Schemas define the interpretation of prim types using structured data. JSON and binary formats define schemas for efficient storage and transmission. Prims have attributes with types and values, allowing for default values and metadata. Attributes, prims, and stages can contain metadata for additional information. This flexibility enables the specification of material properties, environmental conditions, and other annotations for urban visual features.

3.2 Materials and Methods for Urban 3D Visual Features Modeling

This section elucidates the procedures and methodologies employed for acquiring, processing, modeling, and simulating visual features in the urban environment (Fig. 1).

Fig. 1.
figure 1

Urban scene materials acquisition and visual features modeling workflow.

Full size image

This workflow outlines the sequential steps and methodologies employed for acquiring, processing, modeling, and simulating urban visual features. It encompasses data collection, pre-processing, geometry modeling, visual features modeling, utilization of USD standard specifications for organization, and the application of geosimulation techniques integrated into the USD framework. This comprehensive approach ensures the creation of accurate and realistic urban 3D models suitable for geosimulation purposes.

4 Experiments and Results Analysis

4.1 Experimental Settings

To evaluate the effectiveness of the proposed approach for urban built environment visual features modeling using USD standard specifications, a series of experiments were conducted to design the digital twin of a housing estate [1]. These experiments utilized a representative dataset of the urban environment as depicted in Fig. 2. The dataset comprised of geospatial data, including aerial imagery, LiDAR data, and GIS data containing information about buildings, roads, terrain, and vegetation. The four categories of experiments conducted are described in Table 3.

Table 3. USD-Based urban visual feature modeling process experimental phases.
Full size table
Fig. 2.
figure 2

A captured parts of urban visual features modeling workflow using USD approach.

Full size image

4.2 Experimental Results Analysis

The analysis conducted aimed to evaluate the effectiveness, accuracy, and efficiency of the modeling process for urban visual features, while also comparing the results with existing methods used for urban simulation and analysis. In the first experiment, we demonstrated the hierarchical organization provided by USD facilitated modular development and management of the urban environment and the ability to establish layer dependencies. This experiment highlighted the flexibility and user-friendliness of layered composition and referencing, enabling to create complex and realistic urban scenes effortlessly. The second experiment aimed to evaluate the accuracy and visual quality of the geometry modeling and material assignment process. The findings indicated that USD offered a robust framework for creating detailed and realistic 3D models of urban features. The supported geometric representations, such as polygons, NURBS curves, and surfaces, enabled precise shape and structure representation. The assignment of materials and textures enhanced the visual realism of the models. Overall, this experiment confirmed that the proposed approach utilizing USD standard specifications resulted in high-fidelity urban visual feature models. The third experiment focused on integrating the urban visual feature models into a 3D geosimulation framework and assessing the simulation results. The integrated model successfully simulated various scenarios, including urban planning, traffic simulation, and environmental analysis. The performance of the model was evaluated in terms of computational efficiency and accuracy. The results demonstrated that the model exhibited real-time interactivity, enabling researchers to dynamically explore and analyze the urban environment. This experiment showcased the suitability of the proposed approach for comprehensive 3D geosimulation applications. In the fourth experiment, the proposed approach was validated and compared with existing methods for urban built environment visual feature modeling. The comparison considered metrics such as computational efficiency, model accuracy, and ease of use. The results indicated that the proposed approach using USD standard specifications outperformed traditional methods in terms of efficiency and flexibility. The ability to iteratively refine the model through layer referencing and overrides reduced manual rework and enhanced productivity. Table 4 provides a summary of the performance and improvements offered by the USD approach compared to current approaches.

Table 4. USD’s metrics compared to current standards for urban visual features modeling.
Full size table

5 Conclusion and Future Work

Urban computing has achieved significant success in addressing various urban problems [40], and urban visual analytics plays a crucial role in empowering urban experts by combining intuitive data visualization and fast computational methods [41]. The research results demonstrated that the USD approach outperformed traditional methods in terms of computational efficiency, model accuracy, ease of use, and productivity, thanks to its iterative refinement capabilities. For future work, several key objectives were identified. Firstly, continual validation and benchmarking of the proposed approach against other emerging urban visual analytics standards would provide further insights and opportunities for improvement. Secondly, integrating USD with data acquisition techniques such as LiDAR, photogrammetry, and IoT sensors could enhance the accuracy and realism of urban models. Thirdly, exploring semantic enrichment and metadata standardization in USD would improve interoperability and enable advanced analysis and simulation capabilities. Fourthly, investigating mechanisms for collaborative urban modeling and data sharing using USD would enable multiple stakeholders to contribute to comprehensive urban models. Lastly, exploring real-time visualization and simulation capabilities using USD would facilitate dynamic interaction and analysis of urban models in various geosimulation scenarios. By addressing these future research directions, the capabilities and applicability of USD for urban built environment visual feature modeling can be further enhanced [37,38,39], contributing to advancements in geosimulation and urban planning. The four journals: IEEE TVCG, CGF, IEEE TITS and ACM TIST, and four conferences: IEEE VIS [42], EuroVis [43], PacificVis [43] and ACM CHI [40], between 2007 and 2022 offer a panorama of very inspiring work in this regard. An interactive tool to explore these articles is available at https://urban-va-survey.github.io.