Keywords

1 Introduction

Educational gaming might seem as very attractive to game developers, mostly because parents are happy to buy games that would help the educational process, however, the revenue that those developers could have is not as high as other categories, such as, competition or adventure. Since the genres often work as intended and as the consumers would buy them for, they work as a certain escape from reality when life becomes too daunting to endure at times.

In the mid-1990s, “edutainment” was a popular concept among teachers and computer-based designers, which is similar to “educational gaming”, but does not imply the same. The first concept goes from what appears to be a regular game, but includes education in it, therefore, the user will learn while completing the game, but the main purpose does not look like the educational process itself. Since the process takes place in the more traditional setting of a classroom which sometimes the children grow to ignore or dislike, the freedom of an open world game makes it a much more enjoyable operation. Also, the word “edutainment” has a second half more akin to entertainment rather than gaming, and that happens to be because at the time, gaming wasn’t either as strong or as marketable as it is today, making it a rather bold idea then. That’s particularly because it was both a growing industry and a rather trialed one for the popularity of some violent games, in the minds of the parents. The second concept, “educational gaming” refers specially to games or apps that are exclusively designed with the purpose of education or learning: they might be fun or interactive, but the goal is clearly established.

Augmented Reality (AR) could be defined as a real-time direct or indirect view of the physical real-world that has been enhanced/augmented by adding virtual computer- generated information and components [1]. AR icons are a representation of the real-life items used for the intended purposes. And as such they’re able to be seen but unable to be touched, because the use or interaction is to be done by the player. Which is precisely the desired experience that the developers wish their users to have.

Since the term “icon” has been around for a long time, there is lack of recent literature defining the term. An icon is a sign which exemplifies its object in a simplified manner [2]. Townsend says that icons should clearly depict, indicate and distinguish commands and operations, and also suggest and indicate the command’s intention [3]. Definitions which the author can get behind as well as adding a bit to them: An icon is and should always strive to be the simplest portrayal of an object in order to be most effectively consumed. Icons are also the smallest part of the visual communication excluding written word which is why they’re so important for our understanding of what surrounds us, whether that is crude reality or augmented reality.

Symbols are those signs which “represent their objects, independently alike of any resemblance or any real connection, because dispositions or factitious habits of their interpreters ensure their being so understood” [4].

In the early 1980s, the availability and growing popularity of personal computers allowed the birth of the consumer software industry. Educators and technologists saw an opportunity to use this new medium to help children learn, and a niche industry of educational software was born (Shuler, 2012). Among the games that were produced and designed, our subject of analysis was created. Carmen Sandiego series first debuted in 1985 [5], created with the idea of a computer game that would get kids interested in geography, but with the main purpose of entertainment. Had it not succeeded, this paper wouldn’t exist. Nor the desire to make it or the AR game prototype which will be elaborated upon in a moment. But since it succeeded, it has become a new avenue for teaching the younger generation about such an encompassing subject as geography is. As it involves the location of places no one would visit if they didn’t know of their existence as well as a bit of their rich history before travelling there.

2 Background and Starting Point

Considering the importance of the Carmen Sandiego series, as it has both a cult-like following as well as many different iterations of all characters involved and the story, they’re in; the author decided to use the game “Where in the World is Carmen Sandiego?”, where the titular character made her first appearance. In this game, the player is a gumshoe detective who must catch the slyest of the thieves, Carmen Sandiego. In all her usual escapades, Carmen leaves various clues the player must decipher to identify Carmen’s next geographic location. Almost like if she was either taunting or testing young detective. The clues are mostly true facts about real geographic locations and deciphering the clues assists the player in learning geographic locales. And right here is where the improved learning process takes place, as the player, in this instance the children, must solve the riddle by themselves in order to progress. The gameplay in Carmen Sandiego is typical of edutainment; drill and practice activities disguised as games [6]. Instead of a different, surreal world, these exercises open the young minds of the children to real places in their own world which they wouldn’t know existed or wouldn’t probably learn of them whilst being so young.

Fig. 1.
figure 1

Original game screenshot

Full size image

As a rather general description, this experiment wants to determine how an old edutainment game interface can be translated to the modern Augmented Reality (AR) design, and how much does it affect the comprehension and understanding of the icons and symbols. The comparison will not only be made with the graphics but will also consider testing methods that are widely known, such as the PSSUQ. It’s also particularly obvious, but the author understands that the graphics need to be at least seeable in order for the child to be immersed in the game and focused on the task at hand, the “cleverly learning” aspect of the game. If the game is visually intolerable, it will obviously not be able to hold their interest and attention in the long run. With all the technological advancements, the quality standards for games have risen dramatically, leaving aside Augmented Reality (AR) games which were barely a genre, let alone almost an industry in the 1990’s.

From the selected game, the study will be focused on the last level, where the player should try to successfully catch Carmen Sandiego, using the clues and following the rules of the game. For this experiment the author considered the same components of the original game, but they had to be updated to the times that we’re living. The thirteenth and last level of the game was purposedly chosen since it is the most complete level of the game as it has all the different mechanics to be learned throughout the playthrough, and the purpose was to evaluate the players having an experience as complete as possible. From the storytelling aspect, the author isn’t worried about the plot being “spoiled” as it’s a well-known franchise ever since its conception. Also, it’s a remake with updated graphics of a pre-existing game and concept. There’s hardly anything else left to “spoil”. Another important aspect is that in this level, the players get to see Carmen Sandiego herself instead of her lackeys as in the other levels. Which also could work as a selling point for this new Augmented Reality (AR) iteration. These lackeys with their generic designs are harder to recognize for the target audience (children 5–7 years old), and this could have led to player biases, to avoid them, an iconic character was chosen. Therefore, the characters used are the ones displayed at the Netflix Carmen Sandiego TV series, mostly because the kids would be familiar with them, these were extracted from a copyright-free source.

Fig. 2.
figure 2

AR prototype screenshot

Full size image

The main components were designed for understanding, if the player would always take the “right way”, therefore, there are not wrong scenarios in this prototype because what was designed was the required for the comparison experiment. In further versions of the game the many different scenarios shall be added to have a more complete game experience with a more open world feel to it.

Even though there were some modifications to give context to the dialogs, most of them remain the same of the original game, because they are not the subject of analysis of the experiment. Any other variables that are not mentioned were not accounted for during the experiment because they were deemed irrelevant.

Fig. 3.
figure 3

Carmen Sandiego appearing in AR version

Full size image

3 Design of the Experiment and Results

The sample of this study consists of 20 participants (60% female, 40% male), aged between 5 and 7 years old (on average, the age is 6.05 years, SD = 0.80). The survey/interview was conducted by the author with the help of a parent/tutor via video call. The average of the self-rated experience with phone on a scale from 1 to 5 was 4.4 (SD = 0.66), being 1 low and 5 high, and having boys (4.25, SD = 0.7) with lower average than girls (4.5, SD = 0.67), there is a detailed table containing all the data in Appendix 1. The reading ability follows the distribution of the Table 1, showing that 75% of the interviewed kids can read:

Table 1. Reading ability
Full size table

On 20 participants aged 5-7 years old.

The evaluated points were:

  1. 1.

    Perceived understanding: “I can understand how this game works”

  2. 2.

    Perceived Usability: “I can play this game easily”

  3. 3.

    PSSUQ (Post-Study System Usability Questionnaire), with the components:

    1. a.

      System Usefulness

    2. b.

      Information Quality

    3. c.

      Interface Quality

To evaluate the Perceived Understanding, and other aspects of the study, some steps were taken:

  • The original game was installed in the author’s personal computer, and TeamViewer (program) access was granted to the parents of the subjects, so they could be able to play the 13th mission of “Where in the world is Carmen Sandiego?”.

  • The access to the prototype was granted to the kids according to their language, meaning that they would evaluate the Spanish or English version with the language that they would feel most comfortable.

  • With the help of the parents/tutor the author taught/gave a demonstration to the players how the AR camera worked.

The scale was from 1 to 7, and the comparison was between the original videogame and the AR version. The perceived understanding of the original videogame decreased after the players completed the test, from an average of 3.2 (SD = 1.12) to 2.95 (SD = 0.66). The participants opinion varied in the AR version: the starting point was 5.6 (SD = 1.2) and increased to 5.9 (SD = 0.88) for the comprehension of how the game works. Figure 1 contains the summary of this category, and Appendix 2 contains the detailed data (Fig. 4).

Fig. 4.
figure 4

Participants perceived understanding

Full size image

The evaluation of the Perceived Usability was done using two different methods. One, using a computer emulator to play the original game and the other, using a cell phone or tablet to play the prototype. The participants had to evaluate how easily they could play the videogames using a scale from 1 to 7, where 1 means strongly disagree and 7 strongly agree. The participants completed the evaluation before and after testing the games.

The results obtained with the Perceived Usability, the values assigned to the original videogame decreased after the players completed the test, and the rate given to the AR version increased after the test was completed. To be more specific, the average values went from 2.9 (SD = 0.85) to 2.7 (SD = 0.64) and 5.6 (SD = 1.2) to 5.9 (SD = 0.99) at the original and the prototype, respectively. Figure 2 is a graphic of the answers of how easily the game can be played, and Appendix 3 is a table containing all the detailed data (Fig. 5).

Fig. 5.
figure 5

Participants perceived usability

Full size image

Before presenting the results of the PSSUQ (Post-Study System Usability Questionnaire), it is important to emphasize how important this questionnaire is in the industry. The PSSUQ has excellent internal consistency (α = 0.80), as well as satisfactory inter-rater reliability (ICC = 0.67). The PSSUQ presents validity, with a high and significant correlation with an overall usability evaluation question (r = 0.84, p < ;0.05). The PSSUQ presents discriminative validity, distinguishing applications with distinct quality [7]. The modifications of the original questionnaire come in the following areas:

  • The questions were modified to help the kids better understand them, specifically, the wording (e.g., instead of “system”, “game”, instead of “how do you explain?”, “what do you think?”).

  • The scale was inverted to have an accurate value, since kids could think that the best would be the highest rates, as well as doing the opposite could lead to fake positive results. Usually, a 1 means strongly agree and a 7 a strongly disagree. Figure 3 summarizes the responses.

Graphically its easy jump to a conclusion, however, we need to evaluate one by one the results of the PSSUQ.

First category: interface quality. It is a measure of the level of comfortability the player has with the things they can see on the screen. More precisely, it is a rate of the quality of the images, sound, functions and capabilities the game has, as well as the level of happiness it provides to the player. The average answer at this category for the original game was 2.9 (being 3 a “slightly disagree” rate; SD = 0.91) and 6.7 for the AR prototype (7 means “strongly agree”; SD = 0.69) (Fig. 6).

Fig. 6.
figure 6

Post-study system usability questionnaire results

Full size image

The obvious dip in quality in this category it’s totally expected to occur because of the age difference. Not talking about the subjects but the games compared with each other. The original used different character models from the intended by the author in their current iteration. And that’s because everything but the plot in and of itself was different at the time of the original’s launch. The images and their color grading were designed in accordance with the hardware’s available capacity at that stage. Now the most recent one, the AR one, was revamped almost entirely by the author and their team to create a more marketable product according to current times.

Second category: information quality. This category takes into consideration the data given by the game, as well as the internal help, hints and how easy it is for the user to find information in said game when it’s needed. Like the other categories, the interviews were performed asking specific questions to the players and watching them play. The average punctuation of the prototype was 6.2 (SD = 0.89) and the original game got 2.9 (SD = 0.95). Even in this apparently unrelated topic, the graphical update makes its presence known. Since the visual quality increased, the readable information also got boosted to match, with that the audio quality grew with the usage of better sound hardware and certain editing techniques.

Third category: system usefulness. How easy and simple the game is to play, the comfortability while playing it and the learning process, were part of the elements evaluated in this category. The players rated section with the help of their parents/tutors, similarly to the others, as well as the guide of the author. The average answers were 2.5 (SD = 0.96) and 5.9 (6 means “agree”; SD = 0.86) for the original version and the augmented reality prototype, respectively. These results are no surprise since the systems in which the games are or would be played, should this prototype see itself being finished and commercialized. In the original iteration, the game was to be played in the available hardware and maybe out of convenience was adapted to recent hardware but the graphics, as previously pointed out, remained the same, faithful to the original. The most recent product of the author and their team will allow itself to be downloaded as an app for smartphones or tablets, making it far more accessible than ever before. As such is way more on with the times and that speaks volumes to its usefulness. On the other hand, the usefulness for the intended purpose, the numbers are to be trusted since the evaluation was conducted with a sample chosen at random.

Chiong and Shuler [8] propose that even though younger children often experience difficulties in using apps on smartphone devices, like uncontrolled swiping, tapping icons incorrectly, accidentally exiting the app and/or not being able “to read” – author’s note – gaming instructions, many of them still find themselves motivated to continue using the device.

This could be noticed in the results of the questionnaire and the other evaluates’ scenarios, the questions of the PSSUQ can be seen on Appendix 4 while the answers are detailed on Appendixes 5 and 6.

4 Discussion and Limitations

The main goal of this study is to determine how an old edutainment game interface can be translated to the modern Augmented Reality (AR) design, and how much does it affect the comprehension and understanding of the icons and symbols. Which has been determined to be an increase so substantial that it makes this a worthy pursuit or at least a considerable endeavor to be partaking. For this, the author translated the icons and symbols to an AR version. The characters used are the ones displayed at the Netflix Carmen Sandiego TV series, mostly because the kids were going to be familiar with them. And that resemblance makes this prototype more likely to be highly marketable, as we human beings have learned to embrace that said familiarity.

The result of the evaluated categories is that the augmented reality game appears to be more visually attractive, easier to play, the player was able to complete it quickly, the provided information was helpful, and the game was clear, pleasant and likeable; when compared to the same game in its original version, that was released to the public in 1985.

There were some limitations in the study and should be acknowledged. The first one is that due to the COVID-19 pandemic that the world is living during the study period, the interviews were done at a distance, using video call technologies. The second one is that part of the data was reported by the parents: even though the author herself did most of the questions and could see what was happening, the parents and/or tutors played an important part. The author does not expect that they would over-report or under-report the players skills or their likeability of the game, but it is worth noting and acknowledging their participation. Since the parents are obviously adults that can convey complex thoughts in order to fulfill the author’s requirements for information during and after the game testing.

5 Conclusions and Further Path

Acquiring new knowledge is a very important part of the development of individuals. Kids nowadays are required to learn more things than all the previous human generations in history, however, this process could be described as boring or not interesting by many students. As well as quite a daunting challenge if taken head-on. Luckily, an edutainment game is a platform where kids can be motivated to complete learning activities, such as games that will provide new areas of knowledge. Because the impetus for making this entire study was to find for more creative ways to make learning as interesting as it could possibly be. That in turn helping the future generations of humanity develop a “continuous learning” mentality, which is of great importance to both face life’s challenges as well as having fun whilst doing it. For life’s challenges it’s what makes it interesting and worthwhile.

The comparison of two different versions of the last level of the edutainment game “Where in the World is Carmen Sandiego?” (Original game versus Augmented Reality prototype) gave a good result for the AR prototype. None of the evaluated characteristics were outperformed by the original version. And even though that was the expected conclusion, it wasn’t totally the intended one, meaning the data wasn’t skewed in favor of one or the other. It just happened to be the case for the reasons previously explained. A detailed comparison between the changes made to the original game can be seen on Appendix 7.

It is difficult to find games that are exclusively designed with the “educational” purpose; however, this study creates a breach where new games could be created as remakes of their original versions. As Charsky said, “there has been a parallel progression from developing edutainment to creating other games” [9]. It is important to review that progression and see that augmented reality can do a lot for successful edutainment games. And how to honor the legacy of the previously great games with an even greater revamp.