Keywords

1 Introduction

Recently, [1] pointed out that gamification research is maturing, transitioning from fundamental “what?” and “why?” questions to more differentiated questions about the implementation of gamification: “how?”, “when?”, and “how and when not?”. In our case, we also encountered a problem without addressing it today in the studies, for example in education, students are analyzed before and during the experience, but not after. In this study, we want to analyze what happens among students when they are no longer gamified, specifically, we analyze 4 aspects: “Pressure/tension”, “Perceived choice”, “Perceived competence”, “Effort/importance”.

Motivation has been studied in numerous papers such as [2, 3]. If we focus on education, we find for example [4], that is about the impact of intrinsic and extrinsic motivation on the participation and performance of undergraduate students in an online gamified learning intervention or [5] which investigated the effects of external rewards on fifth graders’ motivation, engagement and learning while playing an educational game.

After a review of the articles published in recent years, we hardly find any reference to the moment when gamification stops. For example, in the review [6] that discusses “how the effectiveness is influenced when the implementation of Gamification in enterprises is stopped”, mentions [7], a study that examines patterns of user activity in an enterprise’s social network service after the removal of a points-based incentive system. The results of [7] reveal that the removal of the incentive scheme did reduce overall participation. Therefore, we have found a lack of research in motivation at the time that gamification stops in all the areas, including education.

2 Method

The study was conducted in the first semester at the University of Lisbon as part of the database subject. The subject had two weekly sessions, a theoretical-practical session and a practice session. The experience that has been analyzed is about a Moodle course parallel to those face-to-face sessions, over 4 weeks. The course was about the Entity Relationship (ER) and the relational models. In order to encourage student participation, both the test group and the control group were rewarded with part of the final grade if they participated.

Students in this subject are in the second year of computer engineering. Most of the students were men of an approximate age of 20 years. The subject had 200 students, of which 190 were registered. Then, 169 registered students completed the tasks and the survey of the first two weeks. The last two weeks, 113 students completed the experience, so there were 77 dropouts. Due to dropouts, the sample of students in the first survey was larger than in the second. The separation of the students was random, neither sex nor age was considered, since they were mostly men of an approximate age of 20 years.

The students had access to the course out of the hours of the sessions of the subject. The students were divided into two groups. The first group began gamified while the second group was the control group. Both groups had the same tasks to perform. It is intended to analyze what happens when a group of students ceases to be gamified, therefore, after two weeks, the test group ceases to be gamified. In addition to being able to contrast the changes that occur from gamified to non-gamified vs non-gamified to gamified, the control group became gamified at two weeks.

Each week the students had a series of tasks to perform. At the end of the second week and at the end of the fourth week the students conducted a survey. These surveys were the ones that provided us with the data on the aspects to be analyzed. The two tests are based on IMI [8]. Due to the incremental R for each element above 4 for any given factor being quite small [8], we put 4 or more questions for each aspect. “Pressure/tension”, “Perceived choice” and “Perceived competence” are measured using 5 questions and “Effort/importance” with 4. Each question had to be answered using a Likert scale from 1 (not at all true) to 7 (completely true).

For the course gamification, different elements have been used that are within the dynamics, mechanics and components of the gamification [9]. For the inclusion of the elements in Moodle, a plug-in called GameMo [10] has been used. The list of elements is as follows: feedback, badges, points, levels, leaderboard, time Limit, locked content and missions

All these elements were used in order to provide a gamified experience within the course of the Moodle platform used. Additionally, each student could see their profile picture, their level, their experience and the experience needed to access the next level on the main page of the course.

3 Results

As mentioned in previous sections, “Pressure/tension”, “Perceived choice”, “Perceived competence” and “Effort/importance” have been measured using IMI [8] with a scale from 1 (not at all true) to 7 (completely true). In Table 1 we can see the results obtained both in the test and in the control group in the first and second questionnaires.

Table 1. Four aspects evaluated.
Full size table

There is a small difference between both groups in all the factors in the first test. Except for pressure, which is a negative aspect, the test group has a little bit higher rating in all aspects. In the second test, the assessment of the aspects was reduced in both groups except the pressure. The differences between both groups in Pressure and perceived competence in the second test were reduced, while the effort/importance and perceived choice were extended.

At the first test it is possible to see very close values in all the sections between the two groups, except for a wider difference in “Perceived competence”. If we analyze using the t-test, we obtain a significant difference in this aspect when comparing the first test of both groups.

In the second test, if we apply the t-test, we observe that only the differences in “Perceived choice” are relevant to have a p-value below 0.05 and “Effort/importance” is close to being significant with a 0.06. Both values are those in which the control group has less valuation.

In the previous data we can see a decrease in both groups in all aspects except in “Pressure/tension” which is considered negative, it undergoes an increase between the first test and the second. The decrease in “Pressure/tension” and “perceived competence” is greater in the test group, while in the rest of the aspects it is higher for the control group. If we apply t-test we did not find significant differences.

4 Conclusions

The objective of this study is to analyze 4 aspects: “Pressure/tension”, “Perceived choice”, “Perceived competence” and “Effort/importance” of the students when the students cease to be gamified. This study has been carried out at the University of Lisbon with students in the second year of computer science engineering. A test group and a control group have been used to contrast the results obtained.

The first test carried out at two weeks, revealed that there were no significant differences between the gamified group and the control group, except in the aspect of perceived competence. The similarity may be due to the novelty produced by the Moodle course to students regardless of the methodology. In the second test, it reflects a reduction in the positive aspects and an increase in “Pressure Tension” both in the test group and in the control group, but without having significant differences in all aspects in both groups. This small reduction may be due to the loss of the novelty effect among the students mentioned above.

If we focus on the change that occurs between the first test and the second one where the experimental group is no longer gamified, a decrease in all aspects is detected. However, this also occurs in the control group, so it is not possible to identify, in this experience, that the cessation of gamification has affected students.

Therefore, we can conclude that in our experiment no significant differences were found in the 4 aspects analyzed when students stop gamifying. As future work, there is a possibility of studying the effects after stopping gamification in experiences of longer duration or focusing on specific elements of gamification.