Keywords

1 Introduction

Measuring and evaluating usability of interactive systems and interfaces is an ongoing challenge for teams that are concerned with improving the user experience [1]. The success of any software depends on several factors and usability is one of the most significant ones [2]. Numerous usability evaluation methods can be used to estimate the usability problems at the earlier stage of a software design in order to overcome the severity level and withdraw the rework of design [1,2,3,4]. Nowadays, usability is the core cause of the success of any technical product [2] which if it is hard to use, will most possibly fail at the market level [2].

As we currently are in the era of interfaces and user experiences, it is quintessential to understand whether how we could adapt an interface to several demographics. Is the concept one size fits all suited for children and teens? The museum specific mobile application, Ocean Game, was designed for the Natural History Museum of Funchal (NHMF), Portugal, in order to engage 9–10 years old children with the museum exhibits, and it was first tested with these target users. After discovering the difficulties of the NHMF in attracting teenagers there, we were given a chance to rethink the same concept for teens aged 15–17. In this context, the following questions arose: How does an application for children fit the world of today’s teenagers? What usability issues are relevant to these young people? The adaptation was initiated through the usability tests below.

2 Ocean Game: The Product to Be Tested

The Ocean Game is a children’s gamified experience to guide a visit throughout the Natural History Museum of Funchal, Madeira Island, Portugal (Fig. 1). The experience aims to explore the museum content and collect information about its 13 main creatures using a smartphone. This information is presented in a textual form, describing several scientific curiosities about each animal. The Ocean Game uses proximity beacons and visual cues in the form of stickers in order to signal the presence of digital content. The process is as follows: (i) when children approach the sticker, a short animation appears on the smartphone screen suggesting some key information regarding that animal; (ii) children can explore more information by scrolling through tweet-sized texts; (iii) after reading them, children are prompted to take a picture of themselves together with the current animal; (iv) this allows them to collect an icon specifically created for each individual animal which will be found in their inventory inside the mobile application; (v) the more creatures the children collect, the higher the level they can reach; (vi) each level awards a star as an achievement and a catalyst in order to push their motivation forward; (vii) at the end of the experience, users take inside of the application a quiz consisting of 13 questions about the animals (these questions are tailored to the information that they just learned during the experience); and (vii) after completing the quiz, the children are presented with a digital postcard containing their self-portrait and ranking.

Fig. 1.
figure 1

Print screens of the Ocean Game: curiosity about the animal explained in text and taken selfie above it (left); inventory of collected species (middle); and digital postcard with results of the gamified experience (right)

Full size image

3 Method: Usability Procedure

In this section, we describe the Ocean Game’s usability individual sessions with 20 students, aged 15 to 17, from a local secondary school in Funchal, Island of Madeira, Portugal. In overall, our sample included 13-male students and 7-female students. Their average age was 16,7. Students were enrolled in multimedia and informatics courses. Each participant participated in the usability test in the school premises where we run the gamified experience in a specially fitted classroom, where the beacons with images had been displaced. At the room entrance, users were given a smartphone and a list of tasks. The tasks were the following: (1) open the app; (2) go over the tutorial; (3) input name, email, age and gender; (4) start playing; (5) collect the 13 species; (6) answer the quiz; (7) check your award; (8) go out the app. During the user test, the researcher was observing and taking notes regarding participants’ behaviors. At the end, they were required to answer a survey on a computer as well as to verbally explain their thoughts of what they liked and disliked more and about their views regarding the selfies.

The surveys delivered were using the System Usability Scale (SUS) [5], translated and validated for the Portuguese language [6], and the Net Promoter Score (NPS) [7]. Our rationale for using these surveys is that they were used also with new mobile technologies [8,9,10]. The SUS and the NPS make it possible to gather personal feedback on system’s quality, and in order to use that feedback for comprehending how the system compares to other alternatives and to identify areas of improvement.

4 Main Findings

In this section, we describe the findings regarding the observations taken by the researcher, the results of the surveys that participants were required to answer, their answers about what they liked and disliked more, and also their thoughts about the selfies.

Observations.

Besides having a list of tasks, we noticed 7 users who were thinking out loud, being confused at the beginning without knowing what to do in the test. In addition, we also noticed that some of the participants did not know when to catch the species or how to take a photo. Others asked if it was necessary to take a properly formed selfie. In overall, they were excited with the user testing when they saw a smartphone to interact with, and we registered the following expression. This is too much technology together, and it is absorbing, which expresses their excitement during the test. We also noticed that almost all people tend to click on the species’ animation and also on the species’ icons placed in their digital inventory. This suggests that students tend to explore more the circular icons with animals and are expecting to see more interactions after tapping the icons.

Surveys.

The overall System Usability Scale had the average score of 83.125 out of 100, which is considered as a “good user experience”. The Net Promoter Score had an average of 8 out of 10 which considers the users to be “passively satisfied”.

Like.

Participants were asked to express themselves what they liked more in the application. In total, 52 insights were obtained which we grouped into 7 categories. The categories were: (1) Learning (12 insights): they enjoy reading information about the species which prompted them to learn; (2) Selfies (10 insights): they enjoy taking selfies with the discovered species; (3) Interaction (10 insights): they like the interaction of approaching the species through the beacon; (4) Usage (8 insights): they value the easy utilization of the application such as the system’s easy and fast answers; (5) Quiz (6 insights): they liked to be prompted with a quiz to test their knowledge capacities; (6) Design (3 insights): 3 participants found the application’s design appealing; (7) Gamified experience (2 insights): 2 participants enjoyed the challenge of going further in the gamified experience by achieving stars.

Dislike.

Participants, when asked about what they disliked in the application, came up with 11 insights, which we grouped into 4 categories. Namely: (1) Selfies (4 insights): participants felt embarrassed while taking self-portraits, they would rather take pictures of the species than of themselves; (2) Quantity (3 insights): it was reported that the gamified experience had too few animals; (3) Quiz (2 insights): at least 2 participants didn’t like the fact of being mandatory to answer the 13 questions; (4) Images (2 insights): also 2 people didn’t like the design of our icons arguing that they could be real rather than a vector image.

Selfies.

Regarding the selfies, all the participants were verbally asked about their opinion. We had 16 positive answers and 4 negative answers. Those who answered positively argued that they would take selfies even without the application. They would also value more if the application could add Augmented Reality icons to the selfies and social component as Snapchat does. However, those who did not enjoy the selfies specified that it would be better to take a photo of the species, not a selfie; and at least one argued that it would be boring to take selfies to all the animals and they would rather take to the animals that they liked, not all of them.

5 Concluding Remarks

These findings contribute to the broader literature on gamified experiences for teenagers in the field of HCI, arguing that it is possible to adapt a gamified experience designed for children into one for teenagers, when the first contains elements that are considered as novel approaches by the “new generation” [11] such as mobile applications, taking pictures and interaction through proximity. Our findings highlight that the selfie element catches teens’ attention, although some of them would rather take a picture of the species than taking self-portraits. In fact, our results resonate with the broader literature which indicates that young people today are born into the world flooded by novel technologies [11, 12] because they were comfortable and excited with the user tests. On this view, [12] reports that the “Generation Z” (teenagers in particular) is more and more engaged with open platforms for information sharing, seeing technology as a vital part of their lives. Besides, more studies argue that when working with this age group, emphasis should be placed on producing combined communication policies that connect the use of interactive technologies with the more conventional media channels [11]. In essence, our study highlights how much children and teens value the technology in their museum experiences and would appreciate the integration of playful approaches with the learning goals of museums to have more enjoyable experiences. For the future work, we will iterate the Ocean Game application based on the usability problems that were verified, such as allowing users to tap on the species’ icons, and to give them freedom either to take a self-portrait or simply to take a picture of the species. Although we have not asked them directly about whether the design of the gamified experience was appealing (which is a limitation of this work), only two people stated they would prefer real images rather than our images (Fig. 1). This insight will be taken into consideration when iterating the Ocean Game application. After these improvements, we will commence the user tests inside the premises of the museum, not only using the SUS [5] and the NPS [7], but also using the MUX scale [3] with the aim to address mobility issues which were not addressed in-situ during the above user tests. Furthermore, testing inside the premises of the museum will enable us to address the engagement and visitor experience of the users regarding the gamified experience and the exhibition (repeat play value, relatability with the exhibits) during the museum tour.