Augmented Reality Visualization of Archeological Data

Abstract

One intention of archeology is the documentation and reconstruction of historical development of mankind. The extracted data of an archeological excavation is usually spatial referenced and visualized with the help of maps or geographical information system. Both, paper maps and digital representations have partly complementary strengths and shortcomings in their application. With Augmented Reality, both Systems can be combined and complement each other. This Work presents a concept for augmenting archeological paper maps with 3D models and additional interaction options. Besides the presentation of contents in 3D space for museum visitors, the identified examples of usage include the generation of new contents to support the archeological work on an excavation site. The mobile application ARAC Maps (Augmented Reality for Archeological Content) realizes this concept based on commercially available devices with the Android operation system.

1 Introduction

Augmented Reality (AR) is a technology that extends the real environment with computer-generated content. The history of AR can be traced back to the development of the ultimate display, an AR system with a head-mounted display developed by Sutherland (1968). Progress in computer technology development in the early 1990s led to an expansion of research and development in AR technology, but AR applications remained limited to laboratories or special installations for a long time, mostly due to the high performance hardware required. Mobile augmented reality was finally enabled on a large scale by the development of devices like smartphones and tablets that combine potent processing and graphics hardware with the necessary sensors.

With the help of AR visualizations information can be interactively communicated from a totally new perspective. Archaeology is one of the most promising applications that can be improved by AR presentations. It is the nature of archaeology, that the structures and objects (artifact’s) uncovered by excavations are often incomplete or damaged. Reconstruction of past structures and artifacts is therefore an important task, especially when the results are to be communicated to a general audience. Typical presentation formats in museums range from posters and maps to actual physical reconstructions. However, physical models are costly and are therefore not always viable. More common are reconstructions presented as smaller scale models or the use of rendered 3D models in video presentations. While such models and videos can present relevant information more effectively than 2D depictions, they limit the transfer of knowledge because they are static and not interactive. Interactive 3D visualizations, and especially AR applications, can improve the communication process because they enable interaction and can also directly tie the presented information to the existing site or artifacts. AR systems have many potential benefits: due to the proliferation of high performance smartphones and tablets the required hardware is relatively cheap and many visitors can in fact use their own device. An additional aspect is the simplified navigation compared to purely virtual environments. Because the AR content is spatially linked to physical artifacts the users can employ their everyday navigation skills to explore the content and do not need to learn a new set of controls. Often interesting 3D content such as reconstructions are already available as a result of the archaeological work but only a small portion can be presented using conventional presentation formats. AR thus has high potential to make a larger selection of contents available, to dynamically adapt the presented content to user preferences and to enable user interaction with the presented information. The challenge is to make the implementation of such AR systems practical and inexpensive for the potential content creators, e.g. archaeologists or museum curators.

2 Related Work

Our work presented in this paper relates both to the augmentation of maps and the use of augmented reality in the presentation of archaeological content. While most AR systems aim to augment a physical environment, the augmentation of individual objects in general and paper maps in particular has been studied by several researchers in the past. An even larger application area has been the use of AR techniques within an archaeological context. We can therefore only provide an overview of several projects that are of particular relevance to our system.

ARCHEOGUIDE (Augmented Reality based Cultural Heritage On-site GUIDE) was an EU-funded project to develop an AR system for visitors to an archaeological site (Vlahakis et al. 2002). The project was centered around the idea of providing visitor to an archaeological site with a mobile AR system, consisting of a portable computer, a HMD (head-mounted-display) equipped with a camera and a GPS sensor for positioning. Based on the interests and requirements of the user the system would then propose a route through the excavation site. During the site visit the system superimposes 3D models of reconstructed monuments into the users view, using the HMD, shown in Fig. 1. Users can interact with the presented content through pointing gestures, e.g. to obtain additional audio information on an object of interest. While ARCHEOGUIDE used custom hardware like head-mounted-displays, later projects have also investigated the use of off-the-shelve hardware in similar application settings. The work on AR in archaeology has focused on the presentation of information to museum visitors. The use of AR as a tool to support archaeologists in the field has remained largely unexplored.

Fig. 1

Example of on site augmentation technique, generated by ARCHEOGUIDE

While systems like ARCHEOGUIDE are designed to work in the specific site of interest applications that augment artifacts like maps have the advantage that they can be used in many sites (e.g. museums, classrooms) simultaneously. One of the first systems for augmenting paper maps with the AR technologies was developed by Bobrich and Otto (2002). Their system used an optical tracking system with specific marker patterns that were attached to the paper map to determine the position of the user with respect to the map. The display of the 3D augmentation information was realized with a HMD. For interaction the user had the opportunity to use additional markers within the field of view of the camera in order to trigger predefined actions, as shown in Fig. 2. As the application depended on a desktop PC for the tracking and rendering the mobility of the user was limited.

Fig. 2

Representation of a terrain model with augmented maps (Bobrich and Otto 2002)

Mobile systems for augmenting paper maps were presented by Reilly et al. (2006) for a system that used RFID chips for tracking and Schöning et al. (2006) for a system using marker patterns. Another augmented paper map system, augmenting charts for pleasure crafts was described by Paelke and Sester (2010), using microscopic dot patterns for tracking. Schmalstieg and Wagner (2006) describe a mobile, marker-based application for virtual scavenger hunts within museums. Many of these early works were largely technology demonstrators due to the technological limitations and the requirement for expensive specialized hardware like HMDs. Another limitation was the need to integrate additional infrastructure (like RFID chips) or information (marker patterns) into the map to enable the tracking functionality.

3 Concept and Requirement Analysis

In order to determine the requirements of the prototype, we interviewed archeologists from the Institut Català d’Arqueologia Clàssica. We identified two user groups, the museums visitors and archeologists working on an excavation site, also referred to as experts. For the user group of visitors, we are focusing on communicating information in an intuitive way. This includes the visualization of 3D models and the delivery of textual and visual information to support the knowledge transfer. The requirements of the experts focus on the generation of new data. We have implemented four potential use cases, two for each user group and integrated them into the archeological excavation process.

3.1 Excavation Process

The typical archeological excavation process, as shown in Fig. 3, consists of four main components. This process is by nature destructive and needs accurate documentation to reconstruct the excavated structures later on. The tools of choice for this documentation and presentation are maps, in particular paper maps.

Fig. 3

Archeological excavation process and its products

The prospection of an area with techniques like airborne photogrammetry or geomagnetic surveys leads to maps of potentially interesting excavation sites. During the excavation operations soil layers are removed to uncover the ancient structures beneath. The positions of structures and artifacts found are mapped before the removal from the site. The interpretation of structures and finds leads to reconstructions of buildings and other spatial referenced information, e.g. trade relations between empires. The final step in the excavation process is the presentation of all the data gathered in the previous steps. The data is usually presented in 2D on paper maps or posters.

The two use cases for the user group of visitors fit into the presentation phase of the excavation process. The data collected in the previous step, like reconstructed buildings should be visualized in 3D on the paper maps. Besides the 3D visualization, the knowledge acquisition should be supported by additional information of the visualized models.

For the user group of experts, we focused the generation of new data to support the field work. First, we will implement an annotation function. This will aid the documentation during the destructive and irreversible excavation phase. The second use case will cover the field of generative models. The phase of excavation and interpretation is usually an iterative process driven by assumptions that will need to be proofed through digs at the excavation site. The possibility to create models based on assumptions directly at the excavation site and manipulate them according to new finds can combine these steps in field and speed up the whole process significantly.

3.2 Comparison of Paper and Digital Maps

The shown archeological workflow includes the use of paper maps. While the relationship between paper and digital maps are often denoted as competing, Dymetman and Copperman (1998) describe it as complementary. This derives from their properties. The content of a paper map is fixed and unchangeable, therefore they are considered permanent. Furthermore they are cheap, lightweight and hence easy to transport. Provided in a particular size a paper map may be used by multiple users.

In contrast digital maps are dynamic and can be adapted to various applications. They provide interaction capabilities and may extent the visualization displaying 3 dimensional models. The major problem of digital maps is need for a rather expensive infrastructure. This results in restricted mobility and is susceptible to technical errors.

Still, the discussed archeological use cases will benefit from the advantages provided by digital maps. Considering the museum use case, a map has to provide (additional) information and must be fairly cheap. Paper maps are cheap but the provided information is predefined and fixed. In terms of communication theory this is called a push medium. A proper information provider, however, also includes pull capabilities. Unlike paper maps, digital maps may provide such a feature, where users can interact with the map and acquire additional information of interesting areas. The distribution of information is thus much more personalized, efficient and attractive.

Moreover, digital maps are already widely used in the archeology domain. Computer-aided design (CAD) and GIS are important components of the archeological workflow. Because of their easy handling, paper maps are nonetheless the main map-tool at an excavation site.

4 Implementation

The goal was to develop a practical and cost effective platform for AR applications that work on common smartphones and tablets, thereby making them accessible to large audiences. Due to the advances of mobile hardware in processing power and the integration of custom hardware for 3D rendering we were able to use natural features tracking, avoiding the need to integrate special marker patterns for tracking.

Our prototype is implemented on Android OS and thus runs on a wide range of mobile devices like smartphones and tablets. The image matching to recognize the paper maps as targets for the AR content is based on the Vuforia SDK. This free-to-use toolkit features a fast and reliable image tracking and is compatible with most Android devices. It supports the generation of new image markers with an online web-tool. In case a predefined marker is recognized in the video frame, the pose of the camera relative to the marker is estimated from the distortion of features found. This pose is used to render the 3D scene and display it on the marker.

The prototype supports VRML and PLY-format files for rendering triangulated 3D models as well as simple point sets, as shown in Fig. 4. In order to combine the models with the markers, a central configuration file with XML layout is used, defining a 3D scene on the markers with different model positions and scales.

Fig. 4

Archeological paper map augmented with simple point sets and triangulated models

The prototype features four different functions to support use cases in the field-work and presentation phase of the excavation process. The presentation of 3D models and additional information is targeting the user-group of museum visitors with the focus on knowledge transfer. The functions to generate a street grid or place digital annotations on the paper map targets the group of archeologists and focus on generating new data to support the field-work.

4.1 Layers

The prototype supports VRML and PLY-format files for rendering triangulated 3D models or simple point sets. To organize the models, a layer system similar to a GIS is used. The layers defined in a separate configuration file can hold an unlimited number of 3D models. Organizing models in layers, as shown in Fig. 5, lots of information can be presented easily on the paper map while the users won’t lose track through too much input.

Fig. 5

Layer selection menu

4.2 Model Information

To display additional model information we use HTML files which can be saved locally on the device or as a global accessible web page. This way the information can be changed dynamically without the need of altering the actual application on each device. In case the user touches a model on the screen, the information is shown in a small HTML-based overlay depicted in Fig. 6.

Fig. 6

Additional model info shown as HTML overlay

In order to check if the user touched a model on the screen, the 2D touch position needs to be converted into the 3D scene. In computer graphics this task is usually referred to as picking. The usual approach is to transform the 2D touch-position into a 3D ray, and intersect it with every object. The computational effort of this intersection depends on the number and complexity of the models and is done fully by the central processing unit. This is rather unfavorable, because the tracking already needs most of the computational power. Therefore we implemented a color picking algorithm, where the scene is rendered in background with all models having a unique color, as shown in Fig. 7.

Fig. 7

Color picking with color coded models

After reading the color at the touch position of this scene, the color code is dereference back to the corresponding model. The advantage of this approach is that the computation is mostly done by the graphics hardware, leaving the CPU for the tracking.

4.3 Generative Street Models

A very useful find on a roman excavation site are ruins of ancient streets. Figure 8 shows the street model of a typical roman settlement as a well-organized street grid.

Fig. 8

Well-organized street grid of a roman settlement

Interpreting the already found street structures gives the scientists hints where to find the structures of buildings in between the street grid and therefore another dig is worthwhile. We use this scenario to implement a real-time modifiable street network as an example for generative models in general.

The generation of a street network model consists of five steps depicted in Fig. 9. First, the user defines a polygon indicating the town boundary by touching the desired positions on the display. In a second step, the application calculates the center of the polygon. Third, the two perpendicular main streets called decumanus and cardo going through the center point are added to the street model. Afterwards, the boundary’s bounding box is filled with streets parallel to the main streets building the grid-based street network. Finally the street network is clipped to the initially created boundary.

Fig. 9

Generation of a roman street network model

After the initial generation, the street network’s orientation, translation and grid size can be intuitively manipulated using common multitouch gestures as Fig. 10 illustrates.

Fig. 10

Archeological paper map augmented with a street network model

With such an interactive street model, the archeologist can verify their assumptions and keep them up-to-date right at the excavation site without waiting for the office work.

4.4 Annotations

Pen annotations on paper maps are a well-known application, but they lack some features which digital annotations can offer. First, a pen annotation changes the map content permanently and irreversible. Second, there is only a limited amount of space on a paper map. Yet, the further use of the notes in a spatial context, e.g. a GIS, is problematic, as it is not georeferenced. Digital notes can bypass these problems. The spatially located notes could be easily transferred to a geo-database for further use. At the current state, the application allows to add, display and delete an arbitrary amount of notes on the paper map area. As shown in Fig. 11, the individual notes are visualized as teal colored rectangles on the map.

Fig. 11

Archeological paper map augmented with an annotation

5 Early Experiences and Feedback

The applications usability was tested with a small group of persons representing the non-experts or in other words the user group of visitors. (ISO-9241-11 1998) defines usability as “the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments”. To test these specific usability criteria, we set up a test consisting of four tasks covering all implemented use cases. During the test we logged empirical analyzable data like the time or clicks required to accomplish the tasks. Besides this data, we checked the user’s satisfaction with a questionnaire at the end of the test. The test setting ignores the user group of the experts and therefore can’t give us hints whether the implemented expert functions can give a real advantage over already existing solutions, but we can identify potential problems in the workflow or the ease of use and learnability of the functions.

The analysis of the data revealed minor problems in the field of usability. For example, the users complained about the small and hard to discover menu handle on the right side of the application. As shown in Fig. 12, we changed this menu handle to a bigger one with more contrast to the underlying paper map, dominated by white and grey colors.

Fig. 12

Bigger menu handle with more contrast increased the usability

Another problem arose from the street network generation task. Aiming for expert users this task involves several consecutive actions which have to be carried out by the user in the correct order. The test revealed this task as rather error-prone for non-expert users. The test users often were not aware of the actions needed in order to create the town boundary and how to manipulate the street network in it. Thus, the app supports the user by showing tooltips popping in the screen guiding him through all necessary steps, as shown in Fig. 13.

Fig. 13

Tooltips guiding the user through a task

The analysis of the questionnaire shows an overall positive view from the users. This indicates that the prototype is generally accepted by the users and the use cases, especially the knowledge transfer for the visitor’s user group benefits from the AR visualization.

6 Conclusions and Outlook

This research evaluated the applicability of Augmented Reality in the archeological domain. Potential requirements and corresponding use cases were identified in collaboration with domain experts from the Institut Català d’Arqueologia Clàssica. From those requirements we derived a concept for an AR-based Application, which we implemented in a prototype Android App. The core of the developed concept is the augmentation of paper maps with 3D interactive content.

The 3D visualization and the distribution of additional information are use cases focusing on non-expert users. The implemented interaction modes are the selection of augmented 3D models for additional information presentation and a layer grouping for these models. In contrast, the use cases focusing on expert users include more sophisticated interaction schemes. The developed prototype implements two expert user interaction schemes for generating additional content. First is an annotation mode, allowing expert users to place georeferenced annotations as augmentation on top of a paper map. Second is a street generation mode, enabling the dynamic generation and manipulation of a typical grid-based roman street-network. Both features do not alter the original paper map in any way.

The prototype implementing the described features is called ARAC Maps (Augmented Reality for Archeological Content on Maps) and bases on the Android platform. The conducted evaluation revealed some minor usability flaws. Addressing those problems improved the usability as well as the stability of the prototype.

Nonetheless, the implementation of all identified requirements and ideas was not possible within the context of this research. So our future work involves the support for more data formats other than VRML and PLY as well as texture and level-of-detail support. Furthermore, the findings of an excavation may originate from different time periods. This fact introduces a time layer feature, assigning a time component to each object or entire layers. With reconstructed 3D building models tagged with such a time component the development of historical sites can be easily and interactively comprehended. Moreover the augmentation of the excavation site itself represents a desired feature. This however, lies beyond the scope of augmenting paper maps with digital content. Finally a comprehensive evaluation with archeologists as the targeted expert user group has to be conducted.