Keywords

1 Introduction

Nowadays many attempts for sharing and maintaining cultural contents by using cloud-computing technology have been made all over the world. ICT services and cloud computing make easier users access to cultural contents by providing application services (such as e-Learning, e-Library, e-Lab, e-Course), data services (such as learning materials, library collections and scientific publications) and infrastructure services (such as virtual storage). So, they can be used to narrowing the gap between different areas, gradually eliminate the digital divide, satisfy the people’s cultural needs, improve the construction of a public cultural service system [1, 2].

The European biggest effort to digitize all of Europe’s heritage and to make it publicly available for everyone is the Europeana project [3]. By combining databases from over 3,000 Europeana cultural heritage institutions and providing the access to more than 15 million cultural objects (images, texts, sounds, and videos), it gives a unique online perspective on Europe’s cultural heritage. An increasing number of projects, built using Europeana API, are contributing with technology solutions, cultural services and contents to Europeana. These projects are run by different cultural heritage institutions, and are co-funded by the European Commission’s Content plus programme and the Information and Communications Technologies Policy Support Programme (ICT PSP). Among the others, CARARE is one of the most relevant project [4]. It aims at aggregating contents from the archaeology and architectural heritage making digital contents related to Europe’s unique archaeological monuments and historic sites available to Europeana users. Moreover, the project provides tools and services to support its network of partners to make their digital contents interoperable with the Europeana environment and enables the access to 3D and Virtual Reality contents through Europeana cloud [5].

3D contents brought to European via the CARARE project, are complemented by 3DICONS project [6]. It enhances the contents base available to European users focusing on digital contents that include 3D models and reconstructions, enlarged models of important details and related images, texts and videos. It will also include and re-contextualize in 3D, objects belonging to a monument but presently located elsewhere, for example in museums. The project’s activities also include the conversion of some existing 3D data into formats that are accessible by European users. The 3DICONS enriched Europeana with an unprecedented quantity of high-quality, 3D, well-organized and attractive information about the masterpieces of European architecture and archaeology [7].

In this context one of the most promising technology for enhancing the users experience, during the navigation through cultural heritage digital contents, is represented by the augmented reality. It can provide the user with an integration of digital contents with real images.

Due to the growing integration of powerful compute capabilities in mobile systems and smart-devices (i.e. oculus [8], google glass [9]), the augmented reality has become one of the research areas with higher demand and interest [10]. Nowadays the augmented reality is becoming even more pervasive in several applications such as in cultural heritage for improving tourism experience with a time navigation or with the integration of historic media [11, 12], in medical application to enhance and support surgical decisions [13, 14], in architectural construction, inspection and renovation [15] or to enhance the gaming experience [16].

This work describes how the augmented reality has been used in the cultural heritage context to enrich the scene of a point of interest (POI) with digital information. The implementation of such system and, in particular, the augmented reality applied in outdoor environment, requires that some challenging issues must be addressed such as the detection of a POI and its tracking when the camera moves. At the beginning, the first augmented reality platforms were implemented with some specific hardware solutions, adapted to different environments (e.g. [17–19]). With the grow of the market, mainly driven by the game consoles (i.e. Kinetic [20] and Nintendo Wii [21]), the devices which integrate cameras and sensors for motion detection, became the first augmented reality systems integrated in mobile systems. In that context, although there were some smart phones with supporting AR technology with marker-based algorithms, the solutions in an outdoor environment remained critical, due to hardware power that was still limited [22]. Many of those mobile platforms were not really user-friendly. The Archeoguide platform, that proposed an augmented reality system for the exploration of cultural sites located in the archaeological site of Olympia in Greece, visualizes the missing and artifacts in the damaged areas of site of Olympia overlaying much additional information. The disadvantage of that project was the use of a heavy wearable HDM integrated helmet (Head Mounted Display) that was not quite mobile. Afterwards, the advantages deriving from the increase of computational power embedded in the new generation of mobile systems led to the development of newer and more performing techniques of augmented reality for outdoor environment. In this regard, the image processing algorithms focused to the matching and tracking of the POIs within of the frame set of observation [23–25] represent the most accurate algorithm for detecting POIs. The increase of computational performance in mobile systems and the use of cloud technologies fostered the development of frameworks devoted to easy the implementation of AR applications, such Wikitude [26, 27] and Metaio [28]. These frameworks provide easy tools to build different multimedia applications exploiting the augmented reality techniques.

The main goal of this work concerns the development of a mobile application with augmented reality capabilities, able to enhance the user’s experience while she walks through the streets of a city looking at monuments, cathedrals and more in general at sites with cultural interest, having all the digital contents in his hands [29].

The next section describes the requirements analysis and the software architecture; The Sect. 3 reports the details about the implementation; follows the description of a case study. Conclusions and future work are described in the last section.

2 Architecture

The requirements analysis represents the first phase of the development process, immediately followed by the design of the software architecture. The application is meant for users with very low knowledge of computer science that would like to enjoy the digital contents related to the cultural heritage by means of a common mobile device. For this reason the application must be intuitive, easy to use and it must respond in real time to the user interaction. The following main functional requirements have been identified for the application:

  • Objects detection: the user must be able to easily use the camera equipped in her own mobile device for the purpose of framing a cultural asset and getting additional related digital contents. The application must hence be able to detect objects within the scene.

  • Overlapping of digital contents with the real scene: once an object has been identified, the application retrieves all the related digital contents and provides the user with such information putting the virtual content on top of the real image acquired from the camera.

  • Management of huge amount of data: the application must be able to handle a huge amount of data with several formats, such as texts, images, video, sounds associated to a specific cultural asset.

  • Offline mode: the application should operate also when the connectivity is reduced or absent.

The application must also meet non-functional requirements:

  • Intuitive to use: the user should be able to use the application without reading any user’s manual. The GUI must include intuitive icons and tips

  • Real Time: the application must respond in real time to the user actions.

  • Cross platform: the application should be used on different mobile platform

  • Fault tolerant: the application must correctly handle faults coming from external modules i.e. connection lost or noise images from the camera.

The proposed framework can be divided into two principal subsystems. The first subsystem contains the Mobile components loaded into a mobile device, while the second subsystem represents the Cloud Layer with its support systems. The overall system architecture is shown in Fig. 1. The Mobile subsystem is implemented from the Scene Identifier and the Rest Call-Images Collector modules.

The Scene Identifier module acquires the image from the camera in real time and, by means of the algorithms of Wikitude SDK, obtains the recognition of a particular Point Of Interest (POI) inside the acquired video frame. The recognition is obtained by comparing the POI inside the frame with those ones loaded in the Target Collection File (wtc file) of Wikitude. If the comparison finds some images, then the Scene Identifier module invokes a REST CALL request to the Data Node subsystem, which then returns all the images related to the matching POI in the video scene. All the selected images are also placed into a logical structure, called Images Collector, which acts as a cache space for the mobile application.

Fig. 1.
figure 1

Ovearall architecture of the system.

Full size image

As shown in Fig. 1, the collection of images used for the POI recognition is placed and, continuously updated, in the database of the Data Node. The Data Node module can add or remove images from the database; at the same time it interacts with the Wikitude Server to create or update the Target Collection File (wtc file) used by the Mobile subsystem. In this way, the Mobile device is able to use both the images loaded in the Data Node and the images stored locally in the mobile device for the detection of POIs. The local data cache also grants the use of the application in the offline mode.

3 Implementation

The proposed application uses the Wikitude SDK framework [10] and cloud computing technologies for obtaining an innovative, multi-platform and user-friendly augmented reality solution. Wikitude SDK ensures the integration with the main hardware platforms, i.e. Android, iOS, Smartphone, Tablet, Smart Glasses. It can integrate more cross-platform languages like JavaScript with HTML directly, to obtain a prototype portable on different mobile platforms. Wikitude implements the API calls in multi-platform environment with an SDK development toolkit in JAVA language.

All the API calls can be re-configured and customized according to the application environment. The JAVA language improves the implementation of multi-platform applications, like Android, iPhone and other smart-devices. The decision to adopt the JAVA language was also supported by the ability to integrate the java applets within web pages with dynamic content as JavaScript.

Wikitude SDK provides the tools to manage the dynamic interactions between the API calls of Augmented Reality with the HTML contents. At the same time, the use of JAVA language ensures the integration of the application modules of augmented reality with other Client/Server applications, like the cloud services and the database access.

The Wikitude framework includes a kernel for intercepting a particular POI inside the frame set of the mobile system, using particular matching and tracking algorithms. In this context, Wikitude engine continuously reads and compares in real time the captured frameset with a the model file with .wtc extension. The wtc model file contains all the information for all the POIs that the application should detect. According to the user requests and by the global position of the mobile system, the model file could be generated by the cloud or locally load into the mobile systems. When one POI is detected into the frame set, the Wikitude engine will send API calls for loading all the images from the cloud repository. A virtual slider with navigation buttons and various touch-screen interactions is created. According to the Internet services and with the user settings, the proposed application could be used offline, by using images and internet structures saved in the cache memory of the mobile system. The virtual slider is synthesized with the historical images loaded for a particular POI and it is implemented by using only the Wikitude Toolkit. The development has been realized through the Instrument for Development with Eclipse in Android environment.

For the purpose of our research we have chosen KVM (Kernel-based Virtual Machine) and OpenNebula open source cloud platform [30] as private cloud technology, since it exhibits great features like openness, simplicity and scalability. OpenNebula assumes that physical infrastructure adopts a cluster-like architecture with a front-end, and a set of hosts where Virtual Machines (VM) will be executed. In order to meet the storage needs of Wikitude application, the Ceph software storage platform [31] is used. It is a distributed store object and file system designed to provide excellent performance, reliability and scalability. Its distributed storage solution allows the system to scale horizontally with multiple machines/heads instead of the more traditional methodologies which use centralized heads with large amounts of storage attached to them. Like most distributed storage solutions, Ceph really has 3 main components:

  • Ceph OSDs (Object Storage Devices) i.e. the place where the blocks get stored. A Ceph OSD Daemon stores data, handles data replication, recovery, backfilling, rebalancing, and provides some monitoring information to Ceph Monitors.

  • Ceph MDS (Meta-Data Server) i.e. the place where the meta-data about files and blocks gets stored, especially used for looking up where the blocks are stored;

  • Ceph MON (Monitors) i.e. the component keeping track of the state of the clustering. It is responsible for cluster management, configuration and state.

4 Case Study

In this section the cloud hardware infrastructure hosting Wikitude application is described. Our cluster configuration is composed by six nodes equipped with the following hardware specifications:

  • 2x Intel Xeon Processor E5-2660 v2 (25 M Cache, 2.20 GHz)

  • 256 GB RAM (16 blocks of 16 GB)

  • 1x 200GB SSD IBM S3700 Enterprise

  • 2x1TB HD in Raid 1 hosting the OS

  • 4x3TB HD in JBOD mode (for Ceph Storage Cluster)

  • 3x NIC 1 Gb/s

  • 2x NIC 10 Gb/s

The cluster is configured as a private cloud with OpenNebula 4.10.2 and Ceph v0.87 Giant release (used as a datastore for Virtual Machine images). Each cluster node, running Linux Cent OS 7.0, acts both as a hypervisor node (Libvirt with QEMU/KVM) for the execution of virtual machines and as a Ceph storage node. Each node is configured with 4 Ceph OSD Daemons, one for each 3TB spinning drive; moreover on 3 of the 6 nodes, there is also the Ceph monitoring daemon. In order to augment storage performance, each node SSD is used for reducing random access time and read latency and accelerating throughput; in particular each SSD is partitioned in the ratio of 1:4 to store the journal data of each Ceph OSD daemon, while the object data are stored on the spinning hard disk drives that are subject to limitations on seek time, access time, read and write times, as well as total throughput. Those physical limitations affect overall system performance–especially during recovery, and can be furthermore reduced by using a dedicated drive (2xHD in raid 1 in our case) for the operating system and software, different from the HDs used for the object storage. Regarding the networking setup, one of the 10 Gb/s NIC is used for the accessing Ceph Storage from the VMs while the other 10 Gb/s NIC is used as an intra-cluster network during the Ceph recovery and rebalancing operations for performance and security reasons. One 1 Gb/s NIC serves to connect to Internet, another 1 Gb/s NIC is used for management, and the third 1 Gb/s NIC is used for connecting to other computing resource.

Ceph stores the client data as objects within storage pools. Using a crush algorithm, Ceph calculates which placement group should contain the object, and further calculates which Ceph OSD Daemon should store the placement group. The crush algorithm enables the Ceph Storage Cluster to scale, rebalance, and recover dynamically. Since 4.0 version, OpenNebula provides an interface for Ceph RBDs (RADOS Block Device), which allows registering images directly in a Ceph pool, and running VMs using that backend. In order to assure the correct use of OpenNebula with Ceph the OpenNebula worker nodes are part of a working Ceph cluster, the Ceph pool is available and the Libvirt/KVM is used as hypervisor.

For the test bed, a tablet Nexus 7 Asus with Android Kit-Kat 4.4.4 has been used. In Fig. 2 the result of the temporal/historical interaction is shown.

Fig. 2.
figure 2

Result shows an added images slider, on the bottom of the display, and a historical image overlapped on the real scene

Full size image

In the proposed approach all of the historic images of the cathedral are displayed in the slide bar at the bottom of the screen. The user can select the historic image that will be superimposed on the real one. In the first phases, the images captured directly from the display of a PC have been used; afterwards others test beds will be conducted in real environment. When the user selects an historic image, a pop-up window is shown with the detailed information about the selection. The selected image can be seen in the contextual window and eventually in full screen (see Fig. 3).

Fig. 3.
figure 3

The augmented reality shows an image slider and a temporal label related to the object on the right of the display

Full size image

5 Conclusion

The manuscript treats an innovative system to enhance the edutainment in the cultural heritage with a cloud computing support. The proposed framework implements two principal components: Mobile component and the Data Node module. The Data Node module manages the Rest Call request calls from the mobile device and updates the images collection for detecting new POIs.

The Mobile component detects POIs inside the captured video frame and composes the augmented scene with a virtual image slider. The Mobile component implements the augmented services and communicates with the Data Node for retrieving the images to synthetize the virtual slider. The overall architecture has been implemented around the Wikitude Sdk engine both for the Mobile component and for the Data Node. The prototype of the Mobile component has been implemented through the Instrument for Development with Eclipse in Android environment; while the Data Node module has been performed by using Ceph software storage platform.

Future developments will be conducted by improving the interoperability with different data formats, and by introducing cross platforms technologies in order to support a wide variety of mobile devices, i.e. iPhone and Window Phone.