Keywords

1 Introduction

For the past three decades, tourism research has endeavoured to describe sustainability in the field. Through this effort, researchers were able to formulate the concept of sustainable tourism and have it show positive results by establishing a theoretical foundation and expanding the base of quantitative studies. Sustainability in tourism has been an active research field for the past three decades [1,2,3,4,5]. At the same time, the emergence of mobile information technology as a commodity in our daily life has provided new opportunities for its exploration in the domain of tourism.

The role of mobile technology in tourism has been emphasized and conceptualized under the term smart tourism [6,7,8,9]. Modern high-power mobile devices and mobile internet services allowed the further penetration of mobile technology into daily life [10]. As a result, the importance of ICT in tourism was acknowledged [11, 12].

This work builds on the technical advancement in mobile technologies and exploits Bluetooth localization of mobile devices and RGB-based feature recognition for AR augmentation to develop a mobile tour guide that can supplement the museum visiting experience through mobile location detection and museum exhibit recognition.

2 Background and Related Work

The evolution of mobile tourism applications and services generated a new trend of mobile tour guides with one of its applications being in the domain of museums focusing both on the tangible and intangible cultural heritage dimensions [13, 14]. Those approaches facilitate mobile technology in standalone mode or in combination with stationery ICT technology to provide various forms of mixed reality experiences [15]. Museum tour guides explore mobile technologies to enhance the museum visiting experience and over the years several different approaches have been proposed including ones that blend virtual humans in the physical space [16,17,18,19] and approaches targeted to the provision of cultural information to people with disabilities [20, 21].

In this research work, we implement a cost-effective localization feature for mobile tour guides based on the combination of AR-based scene feature detection and Bluetooth beacons. Bluetooth beacons are used for a rough estimation of the location of the mobile device (e.g. the room it is in) while AR-based localization provides fine-tuning based on the detected features. Together they implement an efficient yet cost-effective localisation approach validated in the context of a museum installation. Among these mobile-AR based tour guides have received a wide attention. From these we can distinguish two main categories. The first category is using AR-based recognition from the mobile device’s camera for localization [22,23,24] and the second is using AR-based recognition for artefact/point of interest recognition [25,26,27,28,29,30].

3 Overview of the Tour Guide

The Venizelos Museum mobile app provides a tour guide experience for visitors of the Museum-Residence of Eleftherios Venizelos in Chania, Crete [31]. Users can select among three different tours, including a complete tour of the premises and a tour focused on children. The application suggests the next exhibit to visit according to the selected tour but also allows the user to freely scan any exhibit’s QR code to view information about it. The details view of each exhibit allows the visitor to view photographs of the exhibit, locate it on the map and read or listen to an extensive description of the item.

Detailed maps of every floor of the museum along with location markers of every exhibit are available, facilitating navigation. The app also communicates with Bluetooth beacons placed around the museum to display on the map the room where the visitor is currently located. If the user has started a tour, the map will only show the exhibits included in the tour, otherwise, the map will display the markers for all the exhibits.

Several exhibits have been enhanced with Augmented Reality features, indicated by physical AR signs next to them. Scanning their QR code or navigating to their details screen through the app enables an AR button that turns on the device’s camera to outset the exhibit’s AR features. Some of the exhibits come to life and start moving, while others show an animation, video or audio related to the exhibit.

The application is multiplatform and runs on iOS and Android phones and tablets. On iOS, it is written in the Swift language [32], using the latest SwiftUI framework [33]. Location beacon communication is achieved using Apple’s iBeacon protocol [34], while Augmented Reality features are powered by the RealityKit framework [35]. On Android, it is written in Kotlin [36], using standard Android mobile development practices. Augmented reality features are powered by the AR Core framework [37]. Both versions of the app share a common JSON data source containing information on exhibits, tours and localized strings in four different languages (Greek, English, German, and French).

4 UI Overview

From the home page of the application, the user has the option to get information about the museum itself or get started with one of the available tours. Among those, the provided variations are long or short, tours that are adapted to younger audiences and free exploration (see Fig. 1, a). Each tour is comprised of several information spots (see Fig. 1, b). Each information point is linked to multimedia information (see Fig. 1, c). While on the tour information points are visualised on the building’s map with the option of filtering them based on the room they are on the floor (see Fig. 1 c, 1 e, 1 f). To support offline visits to the museum this functionality is available both onsite and offsite. When offsite, users can browse information points virtually through the digital map and get more information about what’s available for them to explore.

The activation of information spots is accomplished either by selecting them through the map or by scanning their respective QR code. Points of interest with AR content can be also located through scanning of the room using the camera of the mobile device. Further information is provided by linking the application with the Bluetooth beacons. Using these beacons the application can localise itself within the museum and provide information when entering or leaving a museum room.

Fig. 1.
figure 1

(a) Mobile App home page, (b) Create personalised route, (c) Preview the route on your mobile phone, (d) On-site AR information, (e) Information point details, (f) Accessibility options.

Full size image

5 Evaluation

The application has been evaluated following an expert-based evaluation with usability and domain experts, namely museum curators. More specifically, evaluation was pursued through the heuristic evaluation approach, according to which a small number of evaluators assess the user interface of the application against well-established principles of usable design, the heuristics [38,39,40,41]. The goal of the evaluation is to identify any potential roadblocks or areas for improvement of the user experience, identifying for each problem the usability principle that is violated. A total of five experts assessed the system across three iterations; in each iteration, the evaluators carefully examined the interface and reported the problems identified, while after the evaluation the interface was updated based on the recommendations provided.

Once the system was deemed acceptable in terms of usability, it was installed at the museum, where it has been used until today by more than 2,000 visitors. This section summarizes findings from the expert-based evaluation, as well as from in situ audits observing the system in action, outlining lessons learned that may be useful for designers of AR museum experiences.

More specifically, the following user experience issues we identified:

  • User localization with beacons requires extensive testing to avoid erroneous recognition, which then causes serious problems for the UX.

  • It is important to be able to correctly identify the user’s location and whereabouts in the museum and to provide information through the mobile guide, correctly indicating which exhibits the user has already visited and which are pending to see.

  • Museum maps displayed on mobile phone screens should present information gradually, depending on the zoom level, to avoid information ‘pollution’ resulting in poor usability.

  • Users should be able to deviate from a selected guided tour (e.g. to see an exhibit that seems interesting and get information about it) and resume the tour at their own convenience, continuing where they left off. In our application, this was supported by scanning the QR code next to the exhibit of interest and then selecting the guided tour to resume.

  • Information about the museum exhibits should be available both on the map and as a list (e.g. list of exhibits available in a guided tour) to support different navigation modalities and user preferences.

  • Museum exhibits placed in a glass display case may cause problems with the feature detection algorithms used in AR. In our approach, this was bypassed by utilizing QR codes placed in close proximity to the exhibit, outside its display case, which acted as anchors for determining the correct location to display the AR objects.

  • The AR viewing experience should afford sudden movements of the camera made by users trying to initiate the AR mode, ensuring to the best possible extent that the AR object is not displaced from the user’s screen.

  • In the case that only specific museum exhibits are augmented with AR, this should be clearly annotated not only in the mobile AR application, but also through a physical sign in the museum. Otherwise, visitors may miss the AR experience or try to initiate AR in exhibits not supporting it, thus creating frustration for visitors. In addition, in situ observations revealed that to launch the camera for AR, users sometimes pressed the “Scan QR” button always available in the app’s main menu instead of the dedicated AR button shown only on the AR-powered exhibit’s details screen.

6 Conclusion

In this paper, we provided an overview of a museum tour guide application that provides alternative visiting modes to support seamless information provision to various audiences and visiting periods both offsite and onsite. For user localization, a mixture of Bluetooth beacons and RGB image feature extraction is employed to provide targeted information on the location the user is in the museum and specific exhibits that through QR codes provide information blended with AR features. The application is currently available both for android and iOS-based devices through the respective app stores [42, 43].