Keywords

1 Introduction

1.1 Shallow and Deep Gamification

The most basic distinction between types of motivation as given by self-determination theory was made by Ryan and Deci [1] by their subdivision of motivation into intrinsic and extrinsic forms. Many school activities, due to their obligatory character, require an external reason for the engagement of the student. However, even if an activity is initially externally motivated, inherently interesting properties of it can be experienced, leading to a motivation shift [1].

Gamification describes various techniques for controlling the behavior of users through game elements towards a specific goal, that is, the application of game elements in a possibly non-gaming context [2]. The overriding goal of gamification is to modify the affected activity, which was originally designed for a specific purpose, so that the user feels intrinsically more engaged, thus increasing their intrinsic motivation and commitment.

There are several experiences of gamification in education, using different approaches [3,4,5]. Techniques can be though at two different levels. In a shallow level or thin layer of gamification, the core teaching and learning processes are not substantially changed. There are still lectures, exercises, homework. But the language changes to making quests, crafting items, defeating bosses with the grade given in Experience Points. An example of such an approach applied to undergraduate studies is given in [4]. Another possible shallow gamification technique is to give stars, badges and prizes for activities in the course, use leaderboards, or yet use game-like interface components. Shallow gamification has been the target of some criticism because it can be seen as manipulative and making excessive use of external motivation [6]. In fact, shallow gamification can be seen as a layer that is put above and on top of the core processes, without changing their essence.

In contrast to shallow gamification, deep gamification can be defined as introducing game elements that change the core processes of the activity [7]. A seminal example of that approach was given by Quest to Learn, an innovative school for grades 6 to 12 that started in 2009 in New York City [5], where the whole curriculum was planned using game design techniques. While shallow gamification needs mainly programing and visual design skills, deep gamification uses mainly game design skills, because it is necessary to rethink the activity and design game mechanics at its core.

1.2 Mathematics Trails, MathCityMap and the MoMaTrE Project

A maths trail is a trail where the participants can discover mathematics in the environment at predefined stations [8]. Blane and Clarke [9] were among the first to present the maths trail idea to a broad, scientific audience. Their concern at the time was the popularization of mathematics. Today, we see the benefits above all in the application of mathematics in real, authentic situations in reality, as well as the modeling that precedes the calculations.

The MathCityMap (MCM) project of the Goethe University in Frankfurt combines the idea of mathematics trails with the possibilities of smartphones [10]. The main idea of the project is to give the participants another perception and experience of mathematics. Maths takes place mainly indoors inside the school, inside the classroom. But all human concepts, including mathematical concepts, are based in the perceptual motor system experiences we have while interacting with the world around us [11]. Furthermore, the project addresses the advantages of the Web 2.0, to have users create content, which can be shared and reused. The MathCityMap website (www.mathcitymap.eu) is a portal, where (teachers) users can create tasks everywhere on the world map. These tasks can be published and if successful, they can be combined with tasks done by other users for a maths trail.

The objective is to bring mathematics outdoors more often, for students as well as every citizen. We seek for new forms of getting a mathematical view of one’s neighborhood, of one’s environment, see the questions and problems, which are everywhere, and on top of that, do it digitally with a smartphone.

In order to leave the borders of Germany and to further develop MathCityMap, a consortium was formed and an Erasmus+ Grant obtained, the MoMaTrE (Mobile Math Trails for Europe) project. The main target groups for this project are student teachers, in-service teachers and the public. Our approach contains a platform and a mobile application to create tasks and with these tasks, mathematical trails can be built by everybody, especially teachers. New ways of collaboration should be possible and we want to build a community of active maths trailers, who share their work and help each other to develop things further. Some shallow gamification elements are part of the app today. In this paper, we will present and discuss a study done to measure their effects and we will present ideas how to apply other gamification elements (including some deep gamifications) to the concept of mathematics trails.

1.3 Literature Results on the Impact of Gamification

Dicheva and Dichev [12] conducted a meta-analysis on gamification in education. Compared to the time span of over four years (from January 2010 to June 2014), where 34 papers related to this topic were published, and the time span from July 2014 to June 2015 (one year), the number of published papers has increased to 41. These findings indicate a growing interest in gamification in the area of education. Nevertheless, only about half of the publications can draw a positive conclusion [12].

The intrinsic motivation and experience associated with playing video games can be seen in gamified activities as well [13]. This is why gamification can benefit on a large array of areas when used correctly as is shown in several studies.

For instance tourism, [14] where the presence of game mechanics in a travel-related application and website resulted in an improvement in terms of engagement and made the experience more social and interactive. Another example is the introduction of gamification in higher education classrooms. One of these cases resulted in an increase in approval rating, interaction and attention in the classroom [15]. Other cases indicated positive effects on the engagement of students and a moderate improvement in learning outcomes, see for instance Ibáñez et al. [16] and Santos [7].

The gamification of a training module with fiction also showed a significant improvement of the satisfaction of the trainees. This same study [17] also shows that the declarative knowledge did not differ between the control condition and the gamified condition, but the procedural knowledge scores were higher in the control condition.

2 The Impact of Shallow Gamification Elements

2.1 Introducing Gamification Elements into MathCityMap

Besides many positive observations of students walking a maths trail, two negative observations lead to the addition of gamification elements to the MathCityMap app. Firstly, students often tried to guess answers, if their first attempt was incorrect. Secondly, there seems to be a motivational obstacle to begin working on the tasks, which is expressed by walking slowly to the first task and thus leading to a low ratio of doing mathematics to the time spent on the trail. Before a decision on the type of gamification was taken, we have analyzed the project as suggested by Morschheuser et al. [18] and defined gamification goals.

The mathematics trail activity as supported by the MathCityMap app focuses on secondary school students, who are familiar with using smartphones and apps. A maths trail is usually carried out on an irregular basis e.g. day’s hike or project days. In our approach, students collaborate in groups of three (one is using the MCM app, one is responsible for measuring and the last person has the task to take notes) and walk the trail independently. To complete a maths trail students have to complete each task that is part of the trail. The activity to complete a task is divided into seven sub activities: (1) finding the task’s location; (2) reading the task’s description; (3) collecting data; (4) transforming task into mathematical model; (5) calculating the answer; (6) entering answer into the app and getting feedback; (7) optionally, taking hints and retry. During the steps (1), (2), (6) and (7) students have to use the application. Finally, two gamification goals were defined.

  1. 1.

    Prevent students from guessing answers.

  2. 2.

    Increase intrinsic motivation for working on maths trail tasks (increase the number of tasks completed per hour).

Based on Lieberoth [19], who found shallow gamification in the form of a game-like design to have impact on the participants motivation, the first gamification elements added to the application were simple and fall into the category of shallow gamification. Following his suggestion to “break clusters of game elements down to individual functional units” [19], we have decided to create three different versions of the app. Each version adds further game elements to the application. To prevent guessing, a points system (G1) was added to the task activity, rewarding the students with up to 100 points for a correct answer. Additionally, each wrong answer after the first one decreases the value of the task by ten points. The second gamification approach is the local leaderboard gamification (G2). It augments the points gamification with the possibility to see the score of the user in front and behind you, so that users get into competition with each other. To not frustrate the last ranked group, we have added a computer player, who will be always ranked last (Table 1).

Table 1. Types of gamification in MathCityMap
Full size table

2.2 The Study

In a study with ninth graders, the following research questions were studied in the summer semester 2017:

  • How does the gamification system influence the motivation of the participants?

  • What effects does the gamification system have on the performance parameters (completed tasks per hour, incorrect entries per task) of the 9th graders when working on a maths trail?

Methodology.

196 ninth grade students (97 female, 99 male) from 14 different school classes took part in the tests. They were randomly assigned to the control group G0 (no gamification; N = 48; 19 female, 29 male), experimental group G1 (points gamification; N = 56; 24 female, 32 male) or experimental group G2 (local leaderboard gamification; N = 92; 54 female, 38 male). After a short introduction to the MathCityMap application and the use of measuring tools, the students were handed out a set of materials (smartphone, measuring tools and a paper trail guide) and walked a mathematics trail in groups of three. The trail’s tasks focused on cylinder problems. Finally, they were asked to fill in a standardized questionnaire on intrinsic motivation We designed the intervention as close to real (German) mathematics classes as possible so that findings could be easily applied to the school context. For instance, teachers divided their students into groups of three themselves. Furthermore, we invited teachers and their students for a frame of 90 min that is a typical duration of mathematics classes (two classes with 45 min each). Subtracting the time for the introduction and the time for answering the questionnaire, about 70 min were left to walk the trail. The motivation questionnaire contained a subset of questions of the Intrinsic Motivation Inventory [20]. The questionnaire comprises 21 items that could be answered on a 7-point Likert scale. Additionally, all inputs that the students made (especially the number of completed tasks per hour and the number of incorrect entries) were logged by the smartphones.

Results.

An earlier study indicated that the influence of gamification might be dependent on the gender of a person [21]. That is the reason why, besides the gamification, also the gender of the students was taken into consideration during the data analysis (Table 2).

Table 2. Summary of the results
Full size table

Multiple two-way analysis of variance were conducted on the influence of two independent variables and their interaction effect (gamification, gender, gamification * gender) on (a) the intrinsic motivation score, (b) completed tasks per hour and (c) number of wrong answers per task. Gamification included three levels (G0 – no gamification, G1 – points gamification, G2 – leaderboard gamification) and gender consisted of two levels (M – male, F – female).

  1. (a)

    None of the effects were statistically significant (F (5, 190) = 1.041, p = .395))

  2. (b)

    All effects were statistically significant at the .05 significance level. The main effect for gamification yielded an F ratio of F(2, 190) = 9.417, p < .001, partial η2 = .09, indicating a significant difference between G0 (M = 3.0, SD = 1.5), G1 (M = 2.3, SD = 1.4) and G2 (M = 3.2, SD = 1.3). The main effect for gender yielded an F ratio of F(1, 190) = 6.2, p < .05, partial η2 = .032, indicating a significant difference between female (M = 2.7, SD = 1.2) and male (M = 3.2, SD = 1.5) students. The interaction effect was significant, F(2, 190) = 4.161, p < .05, partial η2 = .042.

  3. (c)

    One effect was statistically significant at the .05 significance level. The main effect for gamification yielded an F ratio of F(2, 190) = 8.056, p < .001, partial η2 = .08, indicating a significant difference between G0 (M = 4.4, SD = 3.6), G1 (M = 4.2, SD = 4.2) and G2 (M = 2.2, SD = 2.4). The main effect for gender yielded an F ratio of F(1, 190) = 2.475, p = .117, partial η2 = .013, indicating that the effect for gender was not significant, female (M = 2.8, SD = 2.9) and male (M = 3.9, SD = 3.9). The interaction effect was not significant, F(2, 190) = 0.181, p > .05, partial η2 = .002.

2.3 Conclusions

The motivation of the groups that used a gamified app version tends to be higher but not significantly so. This result is in line with the analyzes of different studies on gamification by Dicheva and Dichev [12] and suggests that the actual activity is crucial for the motivational expression and the introduction of shallow game elements have at most a small impact on this. On the other hand, the speed of task resolution was significantly influenced by gamification. The leaderboard group G2 stands out with a higher number of tasks processed per hour and less mistakes per task. In particular, boys responded to the competitive nature of the leaderboard [22], addressing and solving an average of 3.9 tasks per hour. Girls work in all three settings a similar number of tasks, but pay attention to the Leaderboard Gamification, especially on the correctness of the results with just 1.9 incorrect entries per task. Zender and Ludwig [23] discuss further results on learning performance of students using the MathCityMap app.

3 MoMaTrE and Deeper Gamification

The above results support the idea of implementing and testing more complex gamification elements. These might not be, by definition, deep gamification elements but they are objectively more rich and elaborated than the ones presented so far. We propose the implementation of two types gamification. One regarding the use of narrative elements to improve the engagement of users and another which adds different types of objectives to the routes, making the use of the MathCityMap a more cooperative and/or competitive experience for the users (depending on the objective).

3.1 Narrative Approach

The addition of fiction elements to the routes in the application can potentially increase the interest and engagement of the students regarding math problems.

A study performed to 858 secondary school students [24] shows that their preference for video games in the classroom is affected directly by, amongst other things, their perception regarding the usefulness of videogames and their previous experiences with them. This could be favorable due to the demographic consisting of students below 18 years old, which recent studies show is more propitious to playing video games [25].

The introduction of narrative elements to the MathCityMap app brings some challenges. Several maths trails which utilize fiction [26, 27] commonly have narrative elements that are highly contextualized and integrated within the tasks themselves, due to the fact that there is only one route.

The MathCityMap app allows users to create and participate in several routes. As such, creating a narrative contextualized for each one is a challenge due to the necessary authoring effort. It is not our goal to force the user who creates a route to also create a narrative that fits said route.

As previously stated, the tradeoff of creating more specialized and contextualized fiction is that the authoring effort is highly increased. Therefore, two possible solutions to cover different approaches for this problem are being worked on:

Non-contextualized Narrative Nodes:

The creation of a set of fiction nodes that between themselves are part of a bigger narrative but that are non-connected with the tasks or the route itself. The nodes could have different purposes:

  • To introduce the narrative (these nodes would appear at the start of a route);

  • To introduce each task;

  • To replace the message of a correct or incorrect answer;

  • To make the bridge between tasks;

  • To conclude the narrative (these nodes would appear after the completion of the last task of the route);

The amount of nodes for each purpose should be enough to avoid repetition inside each route. This solution would be light in terms of authoring effort but at the cost of some disconnection between the tasks and the narrative.

Contextualized Narrative Nodes:

This approach is similar to the one stated above but with more specified narrative nodes to fit the context of the tasks. The narrative would make references to the problems present in the tasks themselves instead of just present the narrative without mentioning the current task objective. This means that more specified nodes would have to be created to introduce a set of types of tasks, for example, nodes could reference some tasks available in the task wizard like:

  • Determination of slopes of ramps;

  • Number of stones in a rectangular wall;

  • Volume of a cylindrical pool;

  • Walk a certain distance or the shape of a geometrical object.

This solution requires a higher authoring effort when compared to the non-contextualized one, due to the fact that each task needs to have specific narrative nodes tailored to it, which implies a larger number of nodes. Also, it may be difficult to create a route where all the tasks have narrative nodes specific to their type. The higher authoring effort can be mitigated with the use of Procedural Content Generation techniques, in a similar way to what has been proposed for game level generation [28].

This approach will allow the tasks to have a more engaging narrative component since the nodes blend better with the tasks and close the disconnection gap between the narrative and the tasks which may exist in the non-contextualized method.

3.2 Gamified Activities

On top of the narrative approach, we propose the addition of objectives to tasks and routes. This also aims to improve the experience of the users and cooperation between them.

As proposed by Hauge et al. [29] context aware activities can be implemented in such a system, for example a Treasure Hunt, or a Conquer activity. Also, the routes can be made to work differently, not showing all the tasks at the start but instead upon the completion of a task, a clue is given regarding the location of the next one or, in some cases, the task itself would result in the finding of the next task. For example: one task may be “walk 50 m south”, and upon arrival at the solution, the next task would be unlocked. This helps connecting the several tasks and making them less individual. On top of that, it also allows for a bigger pool of options in the integration of the narrative elements.

A factor that can be added to an activity is the time taken by the students to complete a route. However, this, along with the remaining variables (like the percentage of correct answers), should be used for the calculation of scores (as opposed to using time as a single factor for score calculation), as seen above. The reason for it is so as to avoid random answers just for the sake of being the quickest, which ultimately would turn the route into a race.

3.3 Global Teams and Challenges

Games are seen as a social activity by most teenagers [30] and this can be used to enhance the experience and engagement of a gamified application.

We propose a feature that creates a light social component, inspired by recent video games like PokemonGO that gives the player the possibility to choose one of three teams to be a part of. These teams do not affect the gameplay in a major way, but give the players something to argue about.

Other example is the Splatfest game mode in Splatoon and Splatoon 2. This mode allows each player to choose one of two options, dividing the community in two teams. At the end of each month, a team win based on the number of players that voted on the team and on the performance of those players in the game. The rewards given to the winning team are not game changing but that does not hinder this event’s popularity.

The proposed feature is similar to Splatfest in the sense that each user of the MathCityMap app could choose a team and from time to time a team would be chosen as the winner based on the amount of completed routes of the members of said team.

This feature does not interfere with the flow of the App making its implementation relatively easy and could help to give a more global and social component to the application.

4 Conclusions

In this paper, we presented the results of a study on the effects of shallow gamification in the mathematical trails application MathCityMap, which is being developed as part of the MoMaTrE Erasmus+ project. The study showed that the introduction of shallow gamification elements have at most a small impact on motivation, as measured by motivation questionnaires, but successfully influenced performance parameters. Especially the leaderboard gamification increased the speed of task resolution and lowered the number of incorrect answers per task. Furthermore, the study indicates that the impact of gamification is gender dependent. So, while the full promised potential of gamification was not reached, there remains the question if deeper gamification can improve these type of results. With the goal of shifting motivation to a more intrinsic nature, we propose to introduce both narrative arcs associated and interlaced with the trails and meta-team and challenge creation. The effects of those gamification techniques in the MathCityMap application will then be tested.