Keywords

1 Introduction

Photos of artifacts do not always provide sufficient information demanded by researchers, students, and enthusiasts who, however, might benefit from interactive 3D models of the objects. Interactive 3D models are already utilized by a number of high-profile museums, such as the Smithsonian,Footnote 1 the British Museum,Footnote 2 and the Victoria and Albert Museum.Footnote 3 These 3D models have aided preservation efforts, broadened public accessibility, and are used for research and education alike [1].

The precise reconstruction of real-world objects requires shape measurements and spectrophotometric property acquisition, typically performed using 3D laser scanners, RGB-D depth cameras, Kinect depth sensors, structured light devices, photogrammetry, and photomodeling [2]. Beyond the dimension and shape of a model, additional information must be captured, such as textures, diffuse reflection, transmission spectra, transparency, reflectivity, opalescence, glazes, varnishes, enamels, etc. Many of these properties can be represented by low-level descriptors [3], which can be used for training machine learning systems for efficient information retrieval [4]. Low-level features, however, when stored as unstructured data, are limited in terms of machine-processability. Also, most low-level descriptors do not provide the meaning of the corresponding 3D scene. These issues can be addressed using Semantic Web standards, so that the structured representation of the corresponding concepts becomes machine-interpretable by linking it to formal definitions and related concepts. High-level descriptors can represent sophisticated concepts and the actual meaning of the visual content, however, they are typically annotated manually, because they are based on human background, knowledge, and experience. Since manual annotation is time-consuming, some software provide the option for collaborative annotation [5].

The formal structured knowledge representation of 3D models enables efficient annotation, segmentation, indexing, and retrieval. Ontology-based 3D model retrieval can be performed not only by textual descriptions, but also by 3D characteristics such as shape or material [6]. The corresponding concepts and properties leverage definitions from OWL ontologies and related concepts from the Linked Open Data (LOD) Cloud [7], and map them to geometric primitives [8]. However, the full exploitation of cultural heritage datasets requires ontology matching to solve heterogeneity issues by discovering semantic links [9].

3D annotations have not been evolving head to head with 3D modeling. The most prominent specification for 3D annotations, Extensible 3D (X3D), is now the industry standard for representing interactive 3D computer graphics in web browsers without proprietary plugins, and is described in ISO/IEC 19775, 19776, and 19777. The standard is supported both by industry-leading proprietary and open source 3D computer graphics software, such as AutoDesk 3ds Max (via a free plugin), AutoDesk Maya, AC3D, Modo, Blender, and Seamless3d. X3D is used directly on web sites as the successor of the Virtual Reality Modeling Language (VRML, ISO/IEC 14772–1:1997) for encoding 3D models and scenes. In 2009, Behr et al. introduced X3DOM, a scalable HTML/X3D integration model to integrate and update declarative X3D content directly in the DOM tree of HTML documents [10].

Developed with XML schemas, the vocabulary of X3D is semi-structured, machine-readable, but not machine-interpretable, making the standard inefficient for automation, data sharing, and data reuse. This issue can be addressed by mapping the application-specific XML schemas of X3D to domain-specific machine-interpretable metadata terms using Semantic Web standards.

2 Related Work

The first OWL-based 3D graphics ontology, OntologyX3D, was created in 2004 by mapping X3D node elements into OWL classes [11]. This ontology, together with upper ontologies, served as the basis for a platform aimed at supporting the development of intelligent, interoperable 3D applications [12]. In the following years, the evolution of X3D resulted in several different approaches to map the standard to structured data. For example, a small subset of the X3D XML schemas was mapped to RDFS by Gilson and Silva in 2006 as part of an ontology mapping aimed at translating SVG to X3D.Footnote 4

In 2012, while attempting to map the X3D XML schemas to OWL, Petit et al. were faced with the consistency issues of the X3D standard [13]. The XSD to OWL transformation was done using XSLT. While the authors created a basic role hierarchy, the resulting ontology was not without flaws. Firstly, it defined property ranges with VRML and X3D datatypes, e.g., SFBool, MFInt32, rather than the globally deployed XML Schema datatypes, such as xsd:boolean and xsd:integer , which would have been a better choice to maximize interoperability. Secondly, the mapping was based on the assumption that X3D nodes are logically equivalent to OWL classes, which did not take ambiguous X3D objects into account. The XLink cross references have been mapped to OWL object properties, and the X3D element attributes to string literals. The ontology defined default property values as RDFS comments. Considering the large share of datatype properties in X3D, this approach did not give satisfactory results for all roles. The mapping also contained typos (e.g., LocalyDefinedType ). At the time of writing, the ontology was not available online anymore.

Not all X3D-based research focus on the improvement of the X3D vocabulary. In 2014, for example, Yu and Hunter developed a 3D annotation software tool based on domain-specific ontologies and X3D terms, using open technologies and specifications, such as W3C’s Open Annotation model, HTML5, jQuery, and WebGL [14]. While the user interface was quite intuitive for annotating cultural heritage objects, the authors mixed semi-structured data with structured annotations by using concepts from the X3D vocabulary directly (without semantic mapping to OWL). The 3D models have been generated using a 3D laser scanner as unstructured data (in VRML), then converted to the also unstructured Polygon File Format (PLY), and finally to semi-structured X3D, which was mixed with structured definitions from ontologies like Dublin Core to provide RDF output, rather than generating structured RDF data right from the start.

3 Ontology Engineering

To address the aforementioned issues, and to cover all the new features added to X3D since the previous OWL mappings of X3D, a novel OWL 2 ontology, the 3D Modeling Ontology,Footnote 5 has been developed with standards alignment and mathematical grounding using description logic (DL) formalism. The X3D terms have been complemented by a large set of new concepts and roles used by industry-leading 3D modeling software, including 3ds Max and Maya.

3.1 Modeling Challenges

During ontology engineering, a number of modeling issues have been identified and resolved, as detailed below. A new namespace structure has been introduced for the OWL 2 translation of the X3D standard with versioning support at http://vidont.org, taking into account the proposed (and not yet standardized) namespace structure, which is http://www.web3d.org/specifications/x3d-version.xsd. The file extension has been omitted from the new namespace structure, and the specifications directory replaced by 3d , i.e., http://vidont.org/3d/, which points to the latest version at any given time.Footnote 6 The web server that hosts the ontology has been set up to serve an RDF file for Semantic Web agents and HTML5 to web browsers using content negotiation, as suggested by the World Wide Web Consortium [15].

Transparent translation of the X3D XML schemas to first-order logic (FOL), description logics, or OWL was not possible, because the X3D nodes and fields do not directly correspond to the unary and binary predicates and constants of first-order predicate logic, nor to the description logic concepts, roles, and individuals, or the OWL classes, properties, and individuals. Furthermore, no relationships are defined for properties in the standard. For this reason, the XSD file of the current version of the standard proved to be inadequate for mapping the X3D vocabulary to OWL 2. This issue has been addressed by manually creating a new concept and role hierarchy. Due to the different aims and designs of its predecessors, no previously released mappings have been used in the proposed ontology. However, terms from general-purpose ontologies, such as Creative Commons, Dublin Core, DBpedia, FOAF, and Schema.org, as well as the domain-specific ontology VidOnt,Footnote 7 have been reused according to Semantic Web best practices [16].

Regarding the naming conventions, X3D properties have a greatly varying scope, as some properties apply to a large group of classes, while others to a particular class only. Also, many X3D terms represent objects that correspond to a class and a property (node/field) at the same time, leading to conceptual ambiguity issues (e.g., color , geoOrigin , TextureProperties ). This was resolved by extending the role names to reflect more specific properties (which can be used even without context) and distinguish them from their concept counterparts. X3D also has consistency issues, such as multiple concepts have homonymous roles, e.g., color can be defined for BlendMode (the constant color used by the blend mode constant) and for ColorRGBA (the set of RGBA colors). This issue was resolved by extending the corresponding terms and declaring all domains with rdfs:domain . The multiple descriptions for a property defined for two different concepts with the same name is misleading in the standard, which was addressed by modifying, and sometimes extending, their descriptions, and declaring them using dc:description . In the X3D standard, there are ambiguous properties that are defined for multiple concepts with a different meaning, are of a different type, or have a different range and domain. For example, the bottom object property of a cubemap defines the texture for the bottom of the cubemap. The bottom datatype property of a cone or cylinder specifies whether the bottom cap of the cone or cylinder is created. This has also been resolved by term extension. Regarding industrial 3D modeling terms, the differences between the terminology of X3D and 3D modeling software have also been considered. For example, bump map declarations used in 3ds Max are equivalent to the shader-specific vertex attributes of FloatVertexAttribute nodes in X3D, such as tangents and binormals.

Some X3D properties correspond to more than one datatype. For example, the axisRotation of CylinderSensor is either a vector or a floating point number. In the first case, axisRotation represents the local sensor coordinate system. In the second, it specifies whether the virtual cylinder’s lateral surface or end-cap disks of the virtual geometry sensor are used for manipulation, or constrains the rotation output of the cylinder sensor. In such cases, two options were considered: either the extension of the property name to be more specific, or the implementation of the less restrictive datatype (applicable only if one of the datatypes is a superset of the other). The description fields have been amended accordingly by adding context to the value of dc:description . The X3D specification features its own datatypes, many of which are based on VRML datatypes (SFNode, SFColor, etc.). Wherever possible, all datatypes have been converted to standard XML datatypes (e.g., x3d:SFBoolean to xsd:boolean , x3d:SFVec3f to xsd:complexType ). The standard declares URLs as strings, which have been declared in the new ontology using the more appropriate xsd:anyURI datatype instead. The majority of X3D properties are datatype properties, many of which have an array of permissible values (representing vectors or matrices) rather than just one value of a specific datatype.Footnote 8 Such datatype properties have been defined as xsd:complexType rather than the more specific but single-value datatypes, such as xsd:float , xsd:integer , xsd:string , or xsd:anyURI . The proposed ontology also features its own datatypes. In contrast to other controlled vocabularies, the X3D vocabulary does not contain individuals, which has been addressed in the 3D Modeling Ontology. Also, the TBox of the proposed ontology is not based solely on the X3D vocabulary; new concepts (e.g., 3d:3DModel , 3d:Dodecahedron , 3d:DesignStudio ,) and new roles (e.g., 3d:animated , 3d:designedBy , 3d:hasVertices , 3d:baseForm ) have also been added to accommodate the needs of 3D graphic designers.

3.2 Modeling Techniques

The scope of the ontology is the knowledge domain of 3D models and scenes; the purpose of the ontology is to provide comprehensive coverage of 3D concepts and roles with X3D alignment. The ontology engineering was based on the in-depth analysis of X3D nodes, fields, and the corresponding semi-structured vocabulary, as well as the manual creation of OWL classes and properties. In contrast to previous RDF/XML mappings, the Turtle serialization was chosen for the proposed ontology, because the XML serialization would have been too verbose for representing the large number of X3D concepts and properties. Because it was infeasible to use automated ontology engineering techniques, such as XSLT transformation, natural language processing-based concept extraction, statistical or machine learning techniques, during ontology engineering, the components of the 3D Modeling Ontology have been individually assessed and manually coded, resulting in code optimality and an easy-to-read layout for future extensions.

3.3 Formal Grounding

The most common variant of the Web Ontology Language, the DL flavor, which is an implementation of a description logic, can be as expressive as \( \mathcal{SROIQ}^{(\mathcal{D})} \) (OWL 2 DL ontologies) [17]. The main benefits of description logics are the higher efficiency in decision problems than first-order predicate logic and the expressivity higher than that of propositional logic. Consequently, description logics are ideal for modeling concepts, roles, individuals, and their relationships. For this reason, the formal grounding of the proposed ontology was written in the form of description logic axioms, and was then translated to OWL 2 (see Table 1).

Table 1. Examples for description logic to OWL 2 translation.
Full size table

3.4 Creating the Concept and Role Hierarchies

The general structure of the ontology is based on the X3D node hierarchy described in the official specification,Footnote 9 however, not all X3D nodes can be transformed directly into a description logic concept or role. As mentioned earlier, some of the nodes are defined as the logical equivalent of both a concept and a role. To eliminate this ambiguity, the proposed ontology applies an extended name and different capitalization for such concepts and roles. Furthermore, the X3D node hierarchy provides a comprehensive concept hierarchy, but is less satisfactory in describing subrole relationships. Due to the number of role overlaps and the different meanings and characteristics of roles sharing the same name, however, the X3D roles cannot be categorized efficiently in a tree structure anyway, and role scopes are indicated mainly by their domain declarations instead.

The first version of the structured X3D ontology covers the entire vocabulary of X3D v3.3, complemented by a new set of concepts and roles. This novel ontology exploits all mathematical constructors of \( \mathcal{SROIQ}^{(\mathcal{D})} \), the description logic underlying OWL 2 DL, which maximizes expressivity and at the same time retains decidability.

3.5 Integrity Checking

The integrity and correctness of the proposed ontology have been checked throughout the ontology engineering process with industry-leading reasoners, including FaCT++ and HermiT. Since the xsd:complexType datatype, which is used extensively in the proposed ontology, is not supported by FaCT++, the final integrity checking was performed using HermiT. String literals have also been checked using a U.S. English spellchecker.

4 Evaluation

The proposed ontology has been evaluated according to the five ontology engineering principles of Gruber, the researcher who introduced ontologies in the context of artificial intelligence [18]:

  • Clarity: the intended semantics of the defined terms are provided in a human-readable form, complemented by machine-interpretable constraints. While the concept and role names characterize the 3D domain, their definition is independent from the 3D modeling context.

  • Coherence: no RDF statement can be automatically inferred from the axioms of the ontology that would contradict any given definition, which has been tested using semantic reasoners.

  • Minimal encoding bias: the conceptualization of the 3D domain has been specified at the knowledge level independent from any symbol-level encoding. The ontology engineering has been conducted using open standards rather than proprietary specifications, serializations, or file formats.

  • Minimal ontological commitment: the proposed ontology has been designed to be as lightweight as possible, and open to more specific implementations.

  • Extendibility: new concepts, roles, and individuals can be easily added to the ontology without changing the core concept or role hierarchy. The proposed ontology features standards alignment and can be easily interlinked with LOD datasets.

The description logic expressivity of the proposed ontology, \( \mathcal{SROIQ}^{(\mathcal{D})} \), is significantly higher than that of other 3D ontologies, making the 3D Modeling Ontology the most expressive ontology to date in the 3D domain (see Table 2).

Table 2. Comparison of 3D ontologies.
Full size table

The ontology files of other 3D ontologies cited in the literature, such as that of the Geometrical Application Ontology [19] or the Common Shape Ontology [20], are not available online anymore, only the corresponding articles and documentation, thus they have been omitted from the comparison.

While geometric and spectrophotometric properties manipulated in a 3D modeling software can be directly represented in any X3D-compliant knowledge representation, only the proposed ontology provides machine-interpretable definitions for these features, as well as data about the modeling software used for creating the model and the geometric primitives that make up the model (see Fig. 1).

Fig. 1.
figure 1

The proposed ontology provides structured representation for 3D models not only with descriptors based on X3D, but also newly introduced descriptors for geometric primitives that constitute a model

Full size image

While the proposed ontology is not the first X3D-based ontology in its class, it is by far the most comprehensive formally grounded structured 3D ontology, and so the most suitable one for interlinking 3D models and model fragments with LOD concepts. Structured 3D model representations correspond to RDF graphs which, when interlinked with LOD resource identifiers, naturally merge to the LOD Cloud. The resulting structured data can be queried and updated using SPARQL [21], which can also combine federated search results retrieved from diverse cultural heritage data sources [22]. Those structured knowledge representations that implement the concepts and roles of the proposed ontology are suitable for not only annotating 3D models, but also reasoning over them, for example, to differentiate between ancient Attican and Apulian red-figure vases.

The semantic enrichment of the above ancient Greek vase model with Linked Open Data, which would not be possible without the proposed ontology,Footnote 10 can extend the structured representation with terms from LOD datasets such as Kerameikos,Footnote 11 the British Museum Collection,Footnote 12 the Art & Architecture Thesaurus (AAT) dataset,Footnote 13 and the Pleiades Gazetteer of Ancient Places,Footnote 14 which yields to

The structured data of 3D models in the cultural heritage domain can be used for creating, using, and sharing historical information about the ancient world—think of museum collection repositories, for example. Structured high-level descriptors are suitable for efficient indexing of declarative 3D models, especially when deployed as lightweight semantics in the web site markup.

5 Conclusions and Future Work

To use 3D models of cultural artifacts to their full potential, machine-interpretable knowledge representation of the 3D models is needed. Rich semantics make efficient indexing and retrieval possible, however, until now no controlled vocabulary or ontology was able to provide complex structured descriptors for 3D models. One of the most comprehensive 3D standards, X3D, is no exception, because it provides a semi-structured vocabulary only.

Early implementations of semi-structured to structured data mapping of the X3D standard came with a lack of formalism, were incomplete, and did not address the inconsistency and conceptual ambiguity issues of the standard. The 3D Modeling Ontology presented in this paper, the most expressive 3D ontology to date, overcomes the above limitations and features the latest standardized X3D nodes and fields as OWL 2 classes and properties with formal grounding and standards alignment. The ontology also extends the X3D concept list with 3D computer graphics software terminology, advancing the OWL 2 mapping of the X3D standard. The integrity of the proposed ontology has been tested with industry-leading reasoners, and the implementation potential evaluated through the semantic representation of 3D models in the cultural heritage domain, one of which has been demonstrated in this paper. The presented ontology is the very first 3D ontology with X3D alignment to provide true Linked Data integration, making it possible to interlink cultural heritage concepts with the LOD Cloud to describe the culture, provenance, and historical era related to 3D artifact models in a machine-interpretable manner, rather than just the vertices and edges of the 3D polygons. As a future work, after the standardization of X3D v4.0 (which is currently in progress), the proposed 3D ontology will be extended with the new concepts and roles of the upcoming standard.