Keywords

1 Introduction

With the progress of 3D technologies, photogrammetry techniques become the adopted solution for representing science-driven data by turning photos from small finds, to entire landscapes, into accurate 3D models.

This paper proposes a module that explicitly couples the photogrammetry process to a semantic knowledge base modeled by our photogrammetry-oriented ontology, ArpenteurFootnote 1. This coupling is represented in form of an export module for AgisoftFootnote 2 to transform the spatial 3D data into a knowledge base modeled by the Arpenteur ontology. This exportation is particularly useful in the pipeline process within our photogrammetry-driven toolbox ArpenteurFootnote 3 for semantic data-lifting: from image gathering to 3D/VR modeling coupled with the knowledge representation by Arpenteur ontology.

This module is based on Semantic Web technologies where ontologies provide us with the theoretic and axiomatic basis of the underlying knowledge bases. In this context, different approaches have been proposed to permit semantic representation and modeling of synthetic 3D content, a state of the art review is detailed in [7].

The paper is organized as follows: Sect. 2 presents our solution for mapping Agisoft python API to the Arpenteur ontology concepts detailing the adopted photogrammetry configuration. Section 3 depicts a use case scenario of exporting and exploiting the Xlendi shipwreck. Finally, Sect. 4 concludes and presents future work plans.

Fig. 1.
figure 1

Mapping Agisoft Metashape python API to the Arpenteur ontology

Full size image

2 Agisoft to Arpenteur Concepts Mappings

The mapping between the two softwares is limited to the generic concept of photogrammetric model as defined in Kraus [8]: photographs, camera, internal and external orientation, 3D points and their observations done onto the photographs. For example feature description and dense cloud are not supported by this mapping.

These two photogrammetry software manipulate similar concepts but of course the translation of digital data from one to the other will have to support some adjustments. For example, the concept of Photograph in the Arpenteur is similar to Camera in Agisoft.

In Arpenteur, a Photograph is the image produced by a camera (film-based or digital) and the Camera is the object that produces the Photograph. This Camera in Agisoft is translated by the concept of Sensor. It should be noted that the concept of Sensor in Agisoft is more complex and will not be fully used, for example it supports the notion of Plane which refers to multi-sensor camera rig approach. This feature is not used in Arpenteur. The Arpenteur Camera Radial Decentring Distortion is so mapped with the Agisoft Sensor Calibration which manages standard internal orientation and lens distortion. 3D points and their 2D projection on images are present of course in both softwares even if they are not modeled in the same way. Hence, an ontology mapping will allow the exchange of information between these two softwares. Figure 1 recapitulates the mapping pattern used for linking Agisoft python API to the Arpenteur photogrammetry concepts.

Radial Distortion Configuration

Since both approaches are based on a standard description of close photogrammetry, it is relatively easy to establish a direct mapping of concepts between Arpenteur and Agisoft. The only remarkably different point is how to describe radial distortion.

Distortion is a physical phenomenon that sometime may greatly impact an image’s geometry without impairing quality nor reducing the information present in the image. Applying the projective pinhole camera model is often not possible without taking into account the distortion caused by the camera lens. So, most photogrammetry software models distortion using the well known polynomial approach proposed by Brown in the 1970s [4]. However, although we have the equations to compensate the distortion, how to compute the inverse function in order to apply such a distortion is not obvious. And this is a crucial technical point, currently most photogrammetric software use the same equations to manage distortion, but some use the mathematical model to apply distortion and other to compensate it and this is exactly the case of the Arpenteur vs Agisoft configuration. To solve this problem of inverse radial distortion, which is not obvious in the context of polynomial approach, some software use an iterative solution [1]. In our case this is not possible: we need to express the polynomial coefficient for Agisoft using the known coefficient used in Arpenteur.

A solution is proposed in [5] and is implemented in this paper. Here are the formula for the four first coefficients \(b_n\) for Agisoft computed from the four coefficients \(-k_n\) used in Arpenteur:

$$\begin{aligned}&\mathrm{b}_1 = -k_1; \quad \quad \quad b_2 = 3k_1^2 - k_2; \quad \quad \quad b_3 = -12k_1^3 + 8k_1k_2 - k_3;\\&\mathrm{b}_4 = 55k_1^4 - 55k_1^2k_2 + 5k_2^2 + 10k_1k_3 - k_4; \end{aligned}$$

3 Xlendi Shipwreck Use Case

We consider the Phoenician shipwreck of Xlendi (Malta) as a use case scenario, where data was gathered by modern photogrammetry techniques presented in a previous work [6] in the framework of the GROPLAN projectFootnote 4. Successive dives on the Xlendi wreck have resulted in several temporal datasets corresponding to seven surveys dates.

An Xlendi artifacts dataset describing the typological and the morphological description of artifacts was published as Linked Open Data presented in previous work [3]. While Xlendi artifacts dataset was introduced manually by archaeologists, the photogrammetric description (camera settings, interior and exterior orientation parameters, extracted and matched 2D/3D points) was automatically exported from the Agisoft software to an ontology file containing the TBox + ABox description. This paper introduces the used exportation module, which is in form of a python script that can be called directly from the Agisoft Metashape sophtware. The script is made available as open source on GitHubFootnote 5.

In the context of Xlendi shipwreck LOD data publishing, we stored the different datasets in an Apache Jena Apache TDBFootnote 6 which is embedded in a FusekiFootnote 7 server. Seven SPARQL GUI user interfaces are made accessible online via our 2D/3D Web tools, allowing to query Xlendi datasets that correspond to the seven survey dates. Listing 1.1 depicts an example of a SPARQL query retrieving the position and the orientation settings of an Xlendi photograph, i.e. the “John_Stills_CC-309”Footnote 8 photograph from the 2018-09-21 datasetFootnote 9.

figure a

The lifted Xlendi datasets are published in a 2D/3D Web representation coupled with the knowledge base datasets in user-friendly web tools available onlineFootnote 10 for querying and semantic consumption of the data, as detailed in [2].

4 Conclusions

In this paper, we introduced a module (python script) for automatic export of the photogrammetry description into a knowledge base dataset modeled by the Arpenteur ontology. This module extends the Agisoft software used in our photogrammetry process. The automatic export is handled by mapping the Arpenteur ontology to the tool’s API. A real data wreck scenario Xlendi was presented in which the photogrammetry process was automatically exported by this new module.

In parallel with the photogrammetry description, we are currently working on the implementation of an ontology-based virtual reality representation to provide a panoramic view (2D/3D/VR) of the data coupled to the semantics knowledge.