Keywords

1 Introduction

As museums move away from being places where exhibits are collected and displayed, they become spaces where people can actively engage in personalized discoveries and challenges [3]. Traditionally, support for museum visits was limited to audio guides and interactive kiosks. However, the fast and wide uptake of interactive and mobile technologies is making audiences more eager to engage in active experiences. Museums are trying to keep up this challenge and enhancing their context to provide more engaging opportunities to interact with the exhibits secured behind glass shelves [4]. Many studies underline the importance of providing museums with improved user experiences both at individual and cooperative levels [5]. Games and narrative elements are proven to ameliorate engagement, motivation and learning within edutainment environments [6]. Moreover, Falk reports on museums experiences that “one size does not fits all” [7], in particular when addressing the “digital natives” generation (currently 15–19 years old), whose beliefs and behaviors are quite different from their previous generations [8]. While museums often offer guided tours for children and adults, very little is designed for the digital natives [1].

Here we present Memories of Carvalhal’s Palace – Haunted Encounters (MoCP-HE), a location-based game which involves players in uncovering factual and fictional elements of a museum premises and collection. We designed MoCP-HE as treasure hunt experience which makes use of AR in order to engage digital native teenagers in pursuing scientific knowledge about the museum exhibits. The goal of this project is to use gaming and narrative elements to influence our target group engagement with the museum content. The experience is currently being studied in order to understand teenage dynamics and preferences in museums providing a way to analyze the complex relationship between context of use, technological solutions, engagement and learning effectiveness. In the rest of this paper we first discuss related work, including museums related location-based games and stories. Then we describe the game mechanics and technological platforms used in Memories of Carvalhal’s Palace: Haunted Encounters. Lastly, we report on some first preliminary user testing carried out with young gamers and users and provide some concluding remarks and indications for future work.

2 State of the Art

2.1 Mobile and Context Aware Guides for Cultural Heritage

The evolution of smartphones and tablets with technologies like GPS, gyroscopes, accelerometers and AR enables the development of a new generation of games with great potential to be applied in location-based experiences [9]. For instance, gamified tour experiences can be tailored to visitors by calculating their position and orientations. Referred to Long et al. [10], as the position-aware handheld intelligent tour, these mobile apps are capable of orienting visitors around public spaces and provide them with custom-fit information [10]. Support and gamified elements of the tour can be accomplished and delivered by interpreting the user behaviour; physical movement within the space and how they interact with the object or artefact of an exhibit, a monument, or another element of interest [11]. The user’s information combined with HCI concepts and location-aware systems can augment the engagement of the user, support their everyday activity and gain more effectiveness in the mobile delivery of information (links, audio messages, images, videos, 3D models) [11]. Exploration and enhancement of environments with 3D and 360VR modelling, allow the exploration of reconstructions of public spaces. Examples include, Mehringplatz neighbourhood in Berlin [12] or the convent of São Francisco in Funchal [13] helping not only in the narrative of the stories but also facilitating acquisition of knowledge in regards to the physical space. In addition, many experiences are designed to work using different platforms that complement the participant interaction within the physical space, making connections between displays, context-aware, storytelling, and historical events [9]. Recently several research projects developed around context-aware experiences and personalising edutainment in cultural heritage grounds. Personal Experiences with Active Cultural Heritage (PEACH), aims at developing educational and entertaining experiences (edutainment) tailored to the user background, needs and interest of the audience [14]. Immersive experiences like 360º Mobile Reality (MVR) are more accessible thanks to the devices affordance and advancement in mobile technology. The rapid growth of these technologies creates new research and design opportunities widely desirable and enjoyable by a wide range of users [14]. Nevertheless, in order to be widely adopted in public settings we need to overcome several design and technical challenges [13]. As a consequence several guidelines for the development of interactive context-aware systems in physical spaces, especially, those devoted to tourism and cultural heritage have been identified: (1) flexibility to enable visitors to explore and learn in their own way, controlling their own pace; (2) content-aware information tailor to their context (personal and environmental); (3) Support for dynamic information – aware of changes in venues, times, menus, opening and closing hours of events, need always to be updated; and (4) Remote access to interactive (online) services [15].

2.2 Serious Games in the Cultural Sector

Several projects in the area of serious games in the cultural sector are supported by museums. Examples include, Yong’s China Quest Adventure [16]; The China Game [17] about Chinese traditions; Fascinating Egyptian Mummies about the spiritual beliefs of the ancient Egyptians; The Great Bible Race [18] dealing with the religious roots of the Western civilization; and, the Mosaica project [19] developed a Jewish heritage game. In the past few decades, a variety of game-based applications were designed for different media platforms and visitor types [20]. Those games are frequently entailed with the goal of using mobile devices to guide families’ explorations through solving mystery and treasure hunting [21, 22].

Cabrera and colleagues [21] built an interactive museum guide called Mystery in the Museum, deployed at historical/cultural museums, which allowed students to play and perform tasks related to certain artifacts while stimulating their imagination. The game mechanics involves exhibits related puzzles such as scrambled images of specific exhibits and verses from manuscripts of the museum. The study revealed that some of the users (13–19 years old teenagers) lost interest in the interactive guide due to the complexity of the tasks, while others switched the focus from the displayed artifacts to the handheld computers.

Ghost Detector [23] is a story-driven location-based museum game for children. In this game, ghosts of various museum artefacts appear on the screen of the visitors’ mobile device and challenges them to find the artefacts that the ghosts are representing. While evaluating this game, children were observed running through the corridors, paying attention to the feedback on the smartphone as well as the artefacts surrounding them. This study highlights how the introduction of the ubiquitous game undoubtedly influenced that level of excitement and engagement with the museum premises.

Intrigue at the museum [24] is a plot-driven mobile game for children structured around exploration and tasks performance. It is a single-player game, and its plot invites visitors to search for a thief in the museum among a set of virtual characters. Clues are given to the players and as they solve riddles after scanning tags deployed in the building. Following a constructivist approach, the game allows children to explore the museum environment freely, according to their interests and agenda. Moreover, AR gaming and storytelling and cultural sector experiments are proliferous and date back to the beginning of the new millennia [25, 26] [ref]. From the related work reported, location-based mobile games seem to represent a relevant learning tool and offer potential for further research and improvements.

2.3 Gaming as a Strategy to Involve Teens in Museum Experiences

While gaming and narrative as strategies to engage young audiences has been extensively studied, the particular application of these techniques to engage teenage public in museum is somehow still in development. Teenagers and participatory design within museum studies are covered by some papers in the Interaction Design Children (IDC) Community. For instance, the study Digital Natives [27] where teens (currently 15–19 years old) collaborated with designers, programmers, anthropologists and museum curators to create four digital installations for an exhibition. The case study Gaming the Museum [28] is another example that started from everyday practices where children’s (14–15 years old) everyday engagement was strong and thus computers games and online communities were chosen to start a process of creating a game for a museum.

According to formal studies about teenage preferences regarding museum engagement, Cesário et al. [29, 30] identified that mobile experience for museum’s teenage audiences should first and foremost include (1) Gaming and storytelling aspects, secondarily teens look for (2) Interaction elements, (3) and Social media connections and last but not least (4) Museum and exhibits relevant information. Based on these findings and in accordance with the museum goals, the authors in collaboration with a team of creatives and developers, designed Memories of Carvalhal’s Palace – Haunted Encounters, an interactive mobile game experience, engaging teens in solving a mystery thorough exploring the museum and its embalmed species collection.

3 The Mobile Game Experience

The Memories of Carvalhal’s Palace – Haunted Encounters is a location-specific non-linear game deployed at the Natural History Museum of Funchal (NHMF). The museum was once the residence of a noble Madeiran family, the Carvalhal’s, before being donated to the city municipality in order to be transformed into a museum. The game builds on the historical backstory of the building enhanced by the embalmed species. The game suggests the premises are haunted by mysterious forces which are disturbing the status quo. At the beginning of the game, two characters ask the player to help them solve the mystery behind the upsetting circumstances in which the museum verses (Fig. 3). Through a game of shadows, the audience is challenged to find and interact with the exhibits displaced around the museum as they match the shadow presented to them on the screen (Fig. 2). By finding the correct animal, the players unlock an AR 3D model of it (Fig. 4, left side) and are prompted with a multiple answers quiz style question that require them to closely check the species in display. As the players progress in their quest, they are rewarded with pieces of a map that will lead them to find the hidden scientific library of the museum. There they are encouraged to look for a treasure chest. inside which they will find the final answers to satisfy their pleading senders (Fig. 5).

3.1 Game Mechanics

The game unfolds encompassing a series of game mechanics which are described below in details.

  • Call for action. Upon launching the app (Fig. 1), two fictional characters – Meara and Isabel – ask the user for help (Fig. 1). Two different museum tours and game endings are delivered, depending on which character the players chooses to help. Meara leads the player to explore the marine species collected the museum, while Isabel steers them to get to know the terrestrial fauna. One character will lead the audience to discover a ghost, who has been haunting the museum for almost a century, while the other will haunt the players herself with a malignant turn of events.

    Fig. 1.
    figure 1

    The opening screen of the Memories of Carvalhal’s Palace game (right side) and a screenshot of the two main characters that call the player for action (left side)

    Full size image

  • Game of shadows. For the player to uncover the truth about the haunting of the museum he/she needs to interact with the species in display on the glass shelves. The players engage in a game of shadows (Fig. 2), where they have to match museum exhibits with the silhouettes proposed by the application, one at the time. By identifying the correct species from its shadow, the players gain some scientific knowledge about the animal and an Augmented Reality (AR) three-dimensional (3D) model of it, which they can manipulate and examine in detail as if it was outside the protected shelves (Fig. 4, left side). In order to identify the correct species for each shadow the game offers a tip that are available clicking on a help button, on the top right corner of the device. As the players answer quizzes and learn about the marine and terrestrial fauna of the archipelago, the audience is rewarded with pieces of a map for each carried out interaction. Once completed the map will finally disclose where they can find the answer to the mystery they have been engaged to solve.

    Fig. 2.
    figure 2

    One of the 12 Rune marker (right side of the figure) used to unlock the 3D model of the Butterfly. Each Rune is related to a different exhibit. This specific rune is related to the Butterfly, as it signifies “change”. Screenshot of the game of shadows interface (left side of the figure).

    Full size image
    Fig. 3.
    figure 3

    Screenshot of the ending of the game according to the two different characters that open the game: Isabel, (left) and Meara (right).

    Full size image
    Fig. 4.
    figure 4

    Left side of the image: user interacting with the 3D model of the Butterfly displayed as Augmented Reality. Right side of the image: two pages of the digital book. It is possible to reload the 3D models of the animals by capturing the rune charms of the book pages.

    Full size image

  • Markers and Augmented Reality Artifacts. Not all the species in display are part of the game. Twelve special markers (six per different characters journey) have been designed in the shape of Runes (ancient Nordic divining alphabet) to contradistinguish the species that are part of the game. When the user finds the animal that he/she recons corresponding to the shadow displayed on the app, he/she can capture the Rune shaped marker (Fig. 2). If the exhibit is the correct one, the user is rewarded with several scientific facts and curiosities about that animal, and a 3D model of it.

  • The AR 3D model artifact – freeing species from the shelves. Once the 3D model is triggered, the audience can manipulate it through an AR interface and explore it in details from all its sides by rotating it in all directions, zooming in into its features, complementing the knowledge gained from the observing the exemplar in the glass shelves with the curiosity generated by manipulating the AR 3D model.

  • Quiz. After capturing the marker and the corresponding AR 3D model, the app asks the audience to answer a quiz related to the taxidermied animal, exhibited in the shelves. The quiz questions are designed to prompt the viewer to look closely at the exhibited species as the quiz is related to some physical details of the animal. After answering the question, either rightly or wrongly, the audience is presented with a text reporting on several additional scientific facts about the animal.

  • The puzzle and the treasure hunt mechanism. For every completed interaction, from finding the silhouette corresponding animal, to receiving the scientific information, the audience is rewarded with a piece of a puzzle, representing a map. Once completed, the map will guide the players to a hidden location, the scientific library of the museum. In the library, the audience is encouraged to look for a small chest, containing the answer to their search for the truth. The chest is a small wooden box hidden among the scientific publications of the library. Opening the chest, reveals a last Rune, which will lead them to two different ending, depending which character they decided to help in the beginning.

  • Conclusion: exposing the haunting. When the user finds the treasure box and captures the last marker, the game is completed. If they had been helping Meara, they will uncover the presence of a restless ghost, once landlord of the place, haunting the museum premises (Fig. 5). If, on the other hand, the players were helping Isabel, they will be met by the evil ghost of Isabel herself (Fig. 5). She will deny them any satisfaction, take all of their points, and warn them not to wander about the museum site as it once was her house, and she doesn’t like strangers walking around!. To quench her wrath and recover their points, the players are given one last option: they need to rightly answer one last question regarding the museum tour.

    Fig. 5.
    figure 5

    Teenagers using the mobile app. to interact with the poster and capture the markers, during the usability test conducted int the lab.

    Full size image

  • The reward, a token for later reflection. In the end, independently from who they have been helping, all the players receive a reward for finishing their quest. They can take a selfie photograph with the encountered ghost and their game punctuation which will be emailed to them. Moreover, they will receive a book (in PDF printable format) containing pictures and scientific facts of the animals that they have interacted with during the tour (Fig. 4). The book also contains the Runa/markers which can be recaptured at any time and release the AR 3D models of the species.

3.2 Implementation

Haunted Encounters makes use of the Unity3D engine, chosen for its flexibility and extensive community support. Some widely known applications that are built upon the Unity3D engine are Pokémon GO and Magic Arena. Unity3D was also chosen for being compatible with the Vuforia Engine, which provides a set of tools such as the use of custom images presented in the real world as in-game triggers, using the custom images’ features for recognition.

The Vuforia Engine allows the recognition of these custom images by uploading them to a database and performing a feature extraction process, determining the quality of the image as a marker for Augmented Reality during the process. This results in a black and white image with emphasis on the highest contrast points. As a rule of thumb, an image with a greater number of features provides for a better target for the AR application. In terms of logic, the bulk of it is in the quiz part, where it follows the behaviour of the Factory software design pattern and builds a list of quizzes. While loading, new instances of the quiz pre-set are created and populated with the correct information relative to the displayed species. Each quiz consists of a set of questions, related to the species to which the marker is associated. To proceed, the player must select the desired species and find the correct image in the physical environment. Vuforia triggers the image recognition event, causing the application to display a question along with a set of possible answers, with a correct answer being worth full score and a wrong one only a fraction of it. Once all the questions have been answered, a new scene will load, tasking the player to find the last image. This will trigger the ending part of the application, starting the playback of videos while showing the surroundings to the player. The video format chosen was webm, for the support of transparency and its light-weight characteristics.

4 Evaluation

4.1 Pilot Testing and Preliminary Findings

Before the formal evaluation the game was piloted with two young girls of the age of 12. The experimenters followed the users during the trial taking notes and collecting loose feedback. An open-ended interview was made to the players to better understand their perception and experience with the game. With this pilot, we aimed at testing the flow of the game, general usability of the interface and gather some preliminary impressions. A summary of insights from the open-ended interview with the young teens is summarized below.

Both users found the game interesting and engaging (“because we played and understood more about the animals.”, C1 and “It was fun to find the animals and take photos” C2). They found the main game challenge (matching the shadow to the correct exhibit and answering quizzes) demanding and referenced the tips as a necessary help (“Not so easy. … the tips helped.” C2). C1 mentioned that she’d like to have more tips and take more photos that could then feature in the game (“we should be able to take a photo when we found the animals and put in the game… More hints instead of only shadow…” C1). One of the girls suggested to add a time challenge to the task of finding the animals corresponding to the shadows (“A time challenge that give us finite time to find the animal corresponding to the shadow” C1). They also appreciated the AR 3D realizations of the animals, and the fact that they could manipulate them (“They were cool” C1 and C2…” Specially because you could touch the screen and manipulate them (turn them around)” C2). The treasure hunt was their top favorite element of the game (“Looking around the museum for the animals, the treasure hunt, looking for the chest, and then the ghost thing.” C2. They positively remembered the ghostly encounter at the end of the game and the game payoff, in the form of the pdf printable book (“We found a ghost it was cool.” C1 and C2. it was cool because we had the book… and could capture the markers again and see the 3D animals. C1). Finally, they mentioned the game enhanced the museum experience (“In this museum, we learned much more than in other museums”, C2, “I was in many museum and this idea is very cool”, C1) and would recommend to friends and classmates (“Yes, but our class is quite troubling and I don’t think they would behave.” C1 and 2).

Preliminary Insights from the Pilot.

The pilot highlighted the positive impact of the game on the young players. They were both highly engaged and had fun, mostly appreciating of the treasure hunt mechanism and learning outcomes. Attention should be paid to balance difficulties of the challenges proposed in the game with the tips given to help solve them. Overall, the flow of the game (the timing, engagement and level of challenge) worked well, but the pilot exposed an interesting addition: a time constrain to some of the challenges. The AR element were also appreciated together with the possibility of re capturing them through the PDF book markers later on. After the pilot study, we proceeded to conduct a more focused evaluation of some of the game features, before deploying an on site study of the game. In the next section, we will describe the usability evaluation carried out after the pilot, while a complete study of the game onsite is being conducted as we speak.

4.2 Formative Usability Evaluation

The game was further evaluated with 16 students from a single school (14 male and 2 female – mean age: 17.25; standard deviation: 1.29). After trying the game the students compiled two different questionnaires: the Multimedia Guide Scale, measuring the reactions to the usefulness and usability of multimedia guides [31], and the AttrakDiff [32] measuring the attractiveness of the experience. Due to the constrained protocols and schedule of the school, the tests were set up in our lab premises where we replicated the control conditions found at the museum. This enabled us to gather data on the general usability, learnability, quality of the interaction and attractiveness of the application.

Procedure.

The users were split into two even groups (8 users per group) who performed the evaluation with the same protocol into two separates but equally sized and equipped rooms. In order to interact with the mobile app., we provided one smartphone shared by every two students. In preparation for the study we photographed and printed A4 sized posters of all the museum exhibits that are covered in the Haunted Encounters game (Fig. 5). These posters contained the picture of the exhibit photographed on the museum shelf and its associated marker on the side. The lab evaluation consisted in the students playing the game in pairs, as they were in the museum, finding the requested exhibit poster and capturing its corresponding marker in order to unlock the content and answer the quiz related to each exhibit. The posters were distributed on a table in the lab room. After the interaction with the mobile app, participants were required to compile two questionnaires one for the Multimedia Guide Scale and the other for the AttrakDiff.

Usability Results.

Within the Museum Guide Scale (MGS) scale, the parameter of General Usability scored an average of 2.57 (median: 2.50) out of 5 points. As lower than average values in this parameter indicate participants not finding usability issues. Hence, results show that some usability issues might be at play in the Haunted Encounters Museum game and its usability could be improved.

The parameter of Learnability and Control with the mobile guide scored an average of 3.58 (median: 3.67) out of 5 points. Higher values in this parameter mean participants understood and could control the multimedia guide. Results from our test indicate that users found the application quite easy and intuitive to grasp, and master.

The parameter of Quality of Interaction with the mobile guide scored an average of 3.90 (median: 3.67) out of 5 points. Higher values in this parameter mean participants enjoyed the quality of interaction of the multimedia museum guide. In our test, the Quality of interaction scored relatively high compared to the other measures. In sum, the participants reported enjoying the experience of interacting with the application, also thanks to the intuitiveness of its feature, which helped them overcome the usability issues encountered.

The overall results from the Attrakdiff scale are very positive: this scale scores from −3 to 3. Hence, values near 3 are strongly positive. In particular, the Pragmatic quality parameters, describing traditional usability and task related to the design aspects, i.e. efficiency, effectiveness and learnability, scored 2.10 (mean 2.14). The Hedonic quality parameters (identity and stimulation), describing aspects of the product connected to the product qualities, such as originality and beauty for example, scored respectively 2.46 (mean: 2.50) and 2.21 (mean: 2.14). The Attractiveness parameter also scored highly positive with an average of 2.58 (mean: 2.64). The perceived Attractiveness of a product results from an averaging of the perceived pragmatic and hedonic qualities.

With further analysis, a significant relationship between Attractiveness and Hedonic qualities (p < 0.003) emerged from the data. Attractiveness was significantly positively related to how well people scored in the Hedonic quality – identity (rs = 0.748, p = 0.001) and stimulation (rs = 0.564, p = 0.023). We can infer that the high attractiveness of our product is strongly related to its hedonic qualities (originality and beauty). On the other hand, there was not any significant correlation between the results from Pragmatic quality and Attractiveness (rs = 0.398, p = 0.127).

5 Discussion

In summary, our pilot and usability evaluations gave us an initial encouraging set of highlights and findings to guide us shape further our game and its future evaluations. In general, the game was well perceived, it was confirmed that it adds value to the museum, and encourage teenagers exploring its content. It was interesting to note how the aesthetics qualities and originality of the game, helped users overcome usability issues, and left players with an enthusiastic impression of the experience. These positive results led us to agree with [6], who argued that games are proven to improve engagement, motivation and learning within edutainment environments. Moreover, our preliminary findings echo broader literature in the field of HCI and gamified experiences for teenagers. Our findings echo [33] arguing that it is possible to have a single gamified experience for teens spanning from 12 to17 years old, when the game contains elements that are considered novel by this teens group, such as mobile applications, gamification, 3D models and interaction through AR. Teens value the technology, and appreciate when it is included in museum experiences. However, we are aware of the limitations of our study: the pilot was conducted with a very limited sample, and the usability study constrained by the reproduction of the experience in the lab, instead of the museum itself. For understanding how this mobile intervention could enhance the user experience of teenage visitors in museums, a deeper analysis is needed. The conducted studies encouraged us to continue the evaluation of the game, in particular exploring the effects of the game tested on site, and with a wider and more gender balanced sample.

6 Conclusions

This article reports on the design and preliminary testing of Haunted Encounters, a teenage targeted mobile game deployed at the Natural History Museum of Funchal. In the game, the audience is called to solve a mystery by looking for and interacting with the museum exhibits, collecting AR 3D models of the species in display while gaining scientific knowledge about such species. The article reports on some preliminary testing of the game and its findings. The game was piloted with two young teenagers, yielding enthusiastic reception and some suggestions on improvements. The game was then tested with further 16 digital native teenagers evaluating its usability and interaction qualities. The results were extremely positive, despite the limitation of the study which was conducted in the lab, instead of the museum premises. The next phase of our work involves extensively testing this prototype inside the museum’s premises in order to validate its flow, game mechanics and overall usability on site. The results from these studies will help us understanding and facilitating the design of engaging interactive museum experiences for the understudied museum teenage audiences.