Keywords

1 Introduction

The Council of Europe defines Cultural Heritage (CH) as a “group of resources inherited from the past which people identify, independently of ownership, as a reflection and expression of their constantly evolving values, beliefs, knowledge and traditions” [7]. UNESCO, on the other hand, promotes both the preservation and enhancement of CH, two activities that can appear as opposite each other. Information and Communication Technologies (ICTs) are being proposed to address these needs, by fostering the enhancement and enjoyment of CH in new ways and, at the same time, by sensitizing people about its importance and fragility [19]. In particular, Augmented Reality (AR) offers opportunities to empower visitors’ experience through the overlay of digital content onto objects, artifacts and environments (e.g. [12, 13]). AR perceived value has been recently investigated from the perspective of different stakeholders [8]: it emerged that cultural organizations, and small museums in particular, often fear the costs of such new technologies and need to have suitable resources in terms of people with technical skills and of hardware/software technologies.

In the Framework Convention of the Value of CH for Society, the Council of Europe also underlines that “everyone, alone or collectively, has the right to benefit from the cultural heritage” [7]. This issue is often neglected in existing ICT solutions, which often present accessibility barriers and ‘one-size-fits-all’ contents; whilst, in the cultural sector, the content to be transmitted might not be the same for a child or for an expert, just like the communication media used for blind and deaf people might be different [2].

The UniBSArt4All project presented in this paper aims at addressing the issues mentioned above according to Universal Design principles. More precisely, this project has been developed around three main themes:

  1. 1.

    User experience. Enhancing user experience of cultural heritage through AR solutions based on automatic recognition of artwork and wireless devices.

  2. 2.

    Accessibility. Making both content and interaction accessible to different types of users.

  3. 3.

    Sustainability. Ensuring long-term sustainability of the solution, both financially and technically, through low-cost technology and a suitable authoring tool.

The project is the result of an interdisciplinary activity carried out by experts in ICT, in CH preservation and promotion, and in accessibility and universal design. In the frame of this project, we developed a cross-platform (Android and iOS) mobile application (UniBSArt4All app in the following), aimed to support users in enjoying museum content in a tailored and accessible way, and a content management system (UniBSArt4All CMS in the following), to be used by museum curators to populate the database of contents exploited by the app.

The system does not require visitors to interact with some invasive or external technology, such as head-mounted displays or smart glasses, but just with their personal smartphones. The solution is thus cheap both for visitors, who must only download a mobile app from the store, and for museum organizations, which must enrich rooms and objects with Bluetooth Low Energy (BLE) devices (beacons) for artwork identification, and possibly pay a service license for image recognition. Most of the effort required to museum curators consists of creating suitable contents for the different types of users that the application is able to support.

The whole system has been preliminarily experimented in a monumental complex belonging to the University of Brescia, in Italy. However, the idea is to provide museum curators with an integrated system supporting the development of tailored and advanced guides for museums.

The paper is organized as follows: Sect. 2 discusses related work about the adoption of AR in cultural heritage domains; Sect. 3 introduces the project UniBSArt4All by describing both the mobile application and the CMS; Sect. 4 briefly describes a preliminary experimentation of the app; while Sect. 5 concludes the paper.

2 Related Work

The adoption of AR is increasing in the CH domain, with specific reference to archeological sites and museums. Digital overlay of information on visitor’s surrounding can be obtained using mobile or tablet devices [11], or smart glasses [20]. These solutions may provide for example 3D reconstructions of artworks and monuments [18, 21], or superimpose digital objects over real ones to allow accessing additional content. Otherwise, a video projector can be used to enrich the real world with digital information that several users may collaboratively enjoy [4]. Interaction techniques for exploring digital information could be moving the device from a layered menu [12], or rotating and shaking the device to send various types of commands. Head-mounted devices, usually adopted to provide Virtual Reality experiences, are an alternative solution to strive for immersion in a museum space with both seeing and gesturing: for instance, TombSeer exploits head-mounted displays to provide an AR experience and combine it with natural gestural interactivity [17]. This requires, however, that visitors wear an external device, provided by the museum, whose operation must be learned easily and whose cost should be kept low.

Our goal is instead to improve visitor’s engagement with cultural heritage through an immersive interaction that is also easy to deploy and to accept by end users: using his/her personal smartphone the visitor can maintain the visual contact with the artwork through the photo camera, and access all additional contents about the artwork in a specific portion of the screen, as well as he/she can interact with these contents in different ways, by adjusting for example its font size or scrolling it or dealing with associated videos or speeches.

Interaction features are however the same provided by usual mobile applications and thus the user has not to learn specific gestures.

User profiling is often used in literature approaches to adapt the CH application to the users’ preferences (e.g., [22]). In our system it is used for tailoring content and interaction experience according to possible user’s disabilities or other characteristics (age and cultural background).

Another novelty with respect to previous solutions is the usage and experimentation of beacons, to allow the automatic detection of artworks, thus overcoming the problems that GPS encounters in indoor environment [11] and avoiding the need of approaching QR codes or fiducial markers in an environment potentially full of people.

Finally, solutions proposed in literature usually consist of ad-hoc systems designed for some specific CH site or museum; no indication about their development and maintenance over time, or deployment over different organizations is in generally provided. Our project started by taking into consideration both visitors’ and curators’ needs, in order to propose a feasible and financially viable solution.

3 UniBSArt4All

UniBSArt4All was iteratively developed starting from scenarios and mock-ups, and by taking constantly into account the three themes mentioned in the Introduction. AR technology was constantly regarded as a new medium of communication with CH and not as an aesthetic experience in itself; therefore, visitors, curators and human contexts remained at the center of the design activity [16].

This activity led to conceive the whole system architecture as shown in Fig. 1. In the following we focus on UniBSArt4All app, which proposes itself as a novel concept of tourist guide, beyond the audio guide [3]. Then, the structure and goals of the CMS will be briefly described.

Fig. 1.
figure 1

The system UniBSArt4All.

Full size image

3.1 UniBSArt4All App

The app was implemented through the Cordova framework, which allows cross-platform development by means of web-based languages. The app is structured along four sections:

  • Homepage that provides first access to the app as well as the last museum news.

  • Museum, which, according to a step-by-step interaction, shows the sequence of floors, rooms, artworks and artwork details (Fig. 2).

    Fig. 2.
    figure 2

    Floors of the museum (a) and map related to the first floor with rooms and related artworks (b).

    Full size image

  • Bluetooth screen, which uses beacon technology to detect the closest artwork and shows information related to that artwork (Fig. 3).

    Fig. 3.
    figure 3

    Bluetooth screen.

    Full size image

  • AR screen, which uses the photocamera and the Wikitude service (https://www.wikitude.com) to recognize the artwork, and shows information related to the recognized artwork (Fig. 4).

    Fig. 4.
    figure 4

    AR screen based on artwork recognition with Wikitude service used in its trial version.

    Full size image

The Bluetooth screen exploits a Cordova plugin that allows scanning the surrounding environment to look for beacons associated to artworks. Whenever a beacon is detected, the ID of the artwork is sent to the application and used to define a proper query to the database for extracting all related contents that will be shown in the right part of the screen; the rest of the screen (2/3) will be reserved to the visualization of information captured by the camera to keep visual contact with the artwork (Fig. 3). Contents on the right might be scrolled and tailored to users’ needs, e.g. fonts may be resized or full-screen can be activated. Whenever more than one beacon is detected, the application produces a vibration and presents the user with a popup including the list of the artworks related to the three closest devices. The list is ordered on the basis of the BLE distance from the visitor, who may then select the artwork he/she is interested in.

The AR screen has been implemented through the Wikitude framework: when the user enters in this screen, a Wikitude World is activated, i.e. a different application, separated from the main application, performs artwork recognition by sending the image captured by the camera to the service made available in the Wikitude Cloud Recognition section. The latter includes a storage where the target images can be uploaded for subsequent comparison with the captured image. When artwork identification is successful, the same application extracts all information from the database and visualizes them in the right part of the screen, similarly to the Bluetooth screen.

The app allows tailoring content and interaction according to the accessibility needs of the user. In particular, the user may select a possible disability condition (hearing or visual impairment), thus allowing the app to adapt its interaction features. For instance, in the case of visual impairment, the app provides voice-over features and organizes the pages in a way suitable to support functionality selection by visually impaired users. In this case, the AR screen is disabled, while the Bluetooth screen provides the indication “Move close to an artwork” (Fig. 5a), which is automatically told by the screen reader available in the device. In case only one beacon is detected, a vibration is produced and then all contents about the artwork are told to the user. Otherwise, a simplified pop-up, suitable to visually impaired users, is shown (and told) to allow artwork selection (Fig. 5b). In case the user declares a hearing disability, proper contents will be loaded that propose videos, possibly enriched with subtitles or descriptions through sign language.

Fig. 5.
figure 5

Interface for visually impaired visitors: artworks detected through beacons (a) and described by the app (b).

Full size image

A second aspect of accessibility is related to the nature of contents, which should be adequate to the age, culture and objectives of the user. In particular, on first access to the app, one may declare him/herself as a child, a tourist or a scholar, and the app will select automatically the contents that are most suitable to that type of user. For instance, children will find artwork descriptions in a simplified language and will have the possibility to vote the artworks they like. The tourist profile is conceived for a user who is not expert in the cultural domain but would like to access more detailed information with respect to a child. Complete and detailed contents, using domain-oriented terminology, will instead be made available to scholars. These different types of content will be created and uploaded in the shared database by museum curators, by means of the CMS.

3.2 UniBSArt4All CMS

A web-based CMS has been specifically developed to support museum curators in the management of the app and its contents (Fig. 6). The CMS has been conceived to allow an easy adaptation of the project to any type of museum. The following sections compose it:

Fig. 6.
figure 6

UniBSArt4All CMS.

Full size image

  • Dashboard: it shows the visit trends in the current year classified according to user profiles.

  • Statistics: it presents several graphics that describe the museum and artwork visits, classified on the basis of temporal intervals that the curator may personalize according to his/her needs.

  • Users: this section allows monitoring user registrations to the app.

  • Museum: this part allows curators to set up the basic data of the app, i.e. the museum floors and their related rooms, as well as to create artwork objects with automatic generation of IDs; furthermore it supports simple assignment of beacons to artworks.

  • Contents: with this section the curators can take news updated, and upload and manage all contents related to artworks, such as photos, videos, textual and audio-descriptions. In particular, videos and descriptions must be properly created for the different user profiles.

In the current version of the system, artwork photos must be uploaded also in the Wikitude Cloud Storage, by inserting artwork IDs in the meta-data, in order to support their recognition through the Wikitude plugin.

4 Preliminary Experimentation

As already mentioned, UniBSArt4All was preliminary experimented in the San Faustino monumental complex of the University of Brescia, and in particular in the cloister of the monastery and in the Tiepolo cell, where one may admire some beautiful frescos. Three technical experimental sessions were carried out to test image recognition under different brightness, prospective and distance conditions, and to test the BLE technology. After some tuning of photo capturing, the app operated successfully. An informal user experiment was finally conducted with four participants (aged 14–25), who tested the app in the field with both Android and iOS. One experimenter told the tasks to be carried out and a think-aloud protocol was adopted to gather users’ comments. The application resulted easy to learn and to use, and only some minor problems emerged, which were easily fixed. All users appreciated the interaction experience with the app.

5 Conclusion

This paper describes the first version of a system supporting the creation of museum apps based on augmented reality and low-cost wireless devices. As to future work, we are planning to develop our own image recognition module, in order to avoid the use of Wikitude services and thus further reducing the usage costs. We are also thinking to include a speech recognition module to make interaction easier, especially for visually impaired people. Other accessibility barriers will also be taken into account in the future, such as cognitive disabilities.

An extended experimentation with several participants and different stakeholders, including people with different disabilities, is obviously necessary to demonstrate the acceptability and feasibility of the system (both app and CMS), and the financial viability of the idea.

We are also planning to extend the personalization feature by considering the possibility for museum curators to create different kinds of profiles according to the specific context, since in the current version the three profiles (child, tourist and scholar) are built-in in the CMS. This will require re-designing the CMS according to meta-design principles [6, 9] and make available end-user development facilities [14, 15] to curators, as discussed in [1, 5, 10]. In particular, as to children (or students in general), different interacting possibilities could be supported on the basis of age, including gamification techniques and serious games.