Keywords

3.1 Introduction

The first wave of COVID-19, in December 2019 and the first quarter of 2020, has prompted institutions throughout the globe to develop creative teaching alternatives to face-to-face conventional learning methods. All educational activities in Hungary were halted on March 13th, and educational institutions were required to adequately manage the transition to hybrid or online education (Sobaih et al., 2021). Budapest University of Technology and Economics (BME) initiated its own coping initiatives, especially in hands on laboratory courses, to ease the transition to online education (Allen & Barker, 2021; Nesmith et al., 2021). In the faculty of Economics and Social sciences (GTK) as well, they started to improve and broaden the usability of Moodle to better organize the classes electronically. Maintaining academic engagement levels during the pandemic waves was the main challenge facing universities worldwide. The COVID pandemic impacted universities all around the globe, compelling many to explore novel techniques to virtual learning in order to maintain students' engagement and productivity levels. Gamification has caught the interest of educational researchers since its rise in popularity in early 2010 (Fui-Hoon Nah et al., 2014; Swacha, 2021). Educational gamification is a technique for improving engagement in an educational setting by introducing gaming elements (Dichev & Dicheva, 2017; Kam & Umar, 2018). In recent years ‘gamification’ has been perceived as a solution for increasing engagement in many fields and aspects such as in business organizations, academic engagement and in promoting sustainable behaviours (Koivisto & Hamari, 2014; Najjar & Salhab, 2022; Xiao et al., 2021). Previous literature also suggests that demonstrated accomplishments by affordances are most typically used in gamification in education and learning (Majuri et al., 2018). Several research have been conducted to investigate the negative impact of pandemics on academic engagement (Babu et al., 2020; Farooq et al., 2020) however literature offers no clear solution for retaining students’ engagement and decreasing dropout rates. Our research is a comparative experimental study that investigates the differences between students’ engagement, dropout rates and performance in two courses in which one is gamified. Research findings also measured students’ satisfaction levels after the successful completion of the courses combined. The courses are of an elective nature and were introduced by the university during the COVID-19 pandemic.

3.2 Literature Review

The rapid growth of internet technologies and the continuous revolution of computer software over the last decade have transformed higher education practices (Hanson, 2009; Tayebinik & Puteh, 2012). E-learning is a subset of technology-based learning that includes websites, learning portals, video conferencing, YouTube, mobile apps and a plethora of other free websites for blended learning tools (Shahzad et al., 2021). Following the suspension of in-person education, worldwide academic institutions used various types of E-learning (Bao, 2020; Abdullah & Abdulla, 2021). Students’ engagement in distance education via e-learning continues to be an intriguing research topic (Kew & Tasir, 2021; Yang et al., 2021) as according to literature research, students of the twenty-first century are considered digital natives (Sharma & Gupta, 2021). Engagement entails students devoting their time and efforts to the study materials displaying learning improvements and interacting meaningfully with others in the class (Dixson, 2015). Students’ engagement is a vital component of high-quality e-learning since it refers to students’ attempt to foster a psychologically committed attitude toward remaining involved in the learning process, especially in obtaining new information and improving critical thinking abilities (Rajabalee & Santally, 2021). Low student engagement levels and high dropout rates are the two major challenges facing e-learning in higher education (Palani et al., 2021; Smaili et al., 2021). Despite the fact that it has been the subject of several studies, dropout rates in e-learning are often greater than those in face-to-face education (Queiroga et al., 2020). In terms of data prediction and analysis, machine learning has improved considerably over the years. This process proved its importance in the educational sphere for monitoring student performance and identifying early disengagement factors (Ciolacu et al., 2017). Many authors examined the educational benefits of gamification (Duggal et al., 2021; Raju et al., 2021) and the emphasis of gamification procedures, which often focus on two goals: Educational learning objectives relating to the content, and fun learning goals connected to the user experiences they produce, such as happiness and fulfilment (Sailer et al., 2017). Game mechanics, often known as game elements, are the components used in and generated from games. The frequently used game elements are as follows: points, badges, leader boards, challenges, levels, rewards, virtual goods, feedback and progress bars (Dichev & Dicheva, 2017; Hamari et al., 2014; Werbach & Hunter, 2012). Game mechanics are fundamental components of gamification that primarily reflect the application aspects found in games (Huotari & Hamari, 2012; Kalogiannakis et al., 2021; Mekler et al., 2013). Over the years and especially during the pandemic, gamification proved its significance in minimizing students’ dropout rates and influencing higher engagement levels (Bouchrika et al., 2021; de la Peña et al., 2021). Moreover, students’ satisfaction level is also a fundamental factor in analysing the efficiency of gamification implementations. Satisfaction has long been researched and defined in literature, and it is most commonly characterized as a comparison between expectations and perceived service quality (Oliver, 1980). Students’ satisfaction has a major influence on the overall success of the e-learning process and contribute to its improvement (Cidral et al., 2018; Yekefallah et al., 2021). Nevertheless, previous literature lacks practical case empirical analysis on students’ behaviour during the pandemic, as literature investigated and focused on one of the gamification effects and neglected others. Our research aims to analyse students’ educational behaviour which will encompass their engagement levels, dropout rates and performance throughout the semester. The research was conducted via Moodle platform which is the e-learning platform chosen by BME. Moodle provides an evolving platform for virtual learning management systems and in consequence Moodle became the logical choice for researchers interested in theories for improving learning and teaching; particularly those related to the controversial concepts of learning styles (Campo et al., 2021).

3.3 Research Methodology

Gamification is a process that intend to change users’ behaviour (Hamari et al., 2014). Almost in all games, points and levels are frequently used. When a players’ game points exceed a certain threshold, the game level increases (Hu, 2020). In order to statistically quantify gamification effects, we chose to include the following gamification aspects in our course: constructive elements (points, levels, leader boards, feedback, time pressure, progress bar and reminders). Furthermore, we included gamification dynamics by incorporating competition between students as well as challenges before passing every level. Our motivational elements included ownership and social recognitions after students passed their levels (Schöbel et al., 2020). The gamified course was divided into different levels and followed a scaffolding mechanism in which learners weren’t granted full access to the course. The aforementioned process emphasized scarcity and unpredictability as physiological motivation tools. Following each level, students were given accomplishment incentives in the form of additional interactive games created using H5P on Moodle. H5P is a free HTML5 Markup Language that facilitates the structuring and presentation of different content on the internet, e.g. videos and interactive elements-based plugin which allows richer content creation in Moodle. Our intention was to improve students’ engagement by granting them ownership of their achievements. The elective course was divided into 13 lessons, 10 of which were theoretical while lessons 11, 12 and 13 were practical. The final practical lesson content enabled us to clearly follow students’ accumulative knowledge during the course. Level completion required students to watch the lecture and then complete a 5-question revision test. Students got tailored feedback based on their performance on the revision assessments. Students received one point for each accurate response (total 5 per test) and were also required to earn a minimum of 3 points in order to progress to the following subject/level. In case of a failed attempt, student had the possibility to repeat the test. To be eligible for the final test, students had to successfully complete all 13 stages/lessons. The final exam consisted of 20 questions, with one point awarded for each correct answer, and the final test passing grade was 9. Although 9 was sufficient to pass the course, students in the gamified course demonstrated higher motivation to achieve higher scores. After successfully finishing the course students were awarded with a personalized certificate. As for the non-gamified course, it was way much simpler to create, as game elements were removed, such as the H5P games, points, progress bar and road map. The course contained the same learning material, same 13 lessons in the same length, order and layout. Both courses contained the exact same teaching syllabus, videos, test questions and optional learning materials (except for the games). The implemented research methodology is experimental, and the collected data was analysed using SPSS to further validate our research hypotheses cited below:

  • Hypothesis 1: Did gamification improve students’ engagement levels?

  • Hypothesis 2: Did gamification minimize students’ dropout rates?

  • Hypothesis 3: Did gamification improve students’ academic performance?

Since there were no appropriate measurement methods described in the literature, we required a more concise understanding by formulating our own measurement tool which was give the name: ‘the learning experience indicator model (Lexi)’. Lexis’ three main indicators: engagement levels, dropout rates and gained knowledge enabled us to fully compare the differences between the two courses (Fig. 3.1).

Fig. 3.1
A chart displays a three-indicator model. They are engagement level indicators, such as video viewing time and chat activity, dropout rate indicators, depicting the point at which students disengage, and knowledge gain indicators, highlighting test results and accumulated points.

Learning experience indicators model

Full size image

Lexis’ main advantage is that it provides a wholesome approach to measure users’ engagement, drop-out rates and gained knowledge. Moreover, the Lexi model can be customized to provide empirical measurement of different types of online courses. Engagement Level is the indicator that determines how active, how involved and interested a student is, and Moodle successfully provided detailed data on students’ active participation. Learning habits were also recorded and illustrated using the E-learning platform. Engagement Level is calculated based on students’ clicks on activities such as watching the course videos, Net Promoter Score (NPS) and their activity in the chat field. Students also had access to optional extra assignments, interactive extra learning materials and essays, which are not required for the successful completion of the course. In terms of dropout rates, we had to contend with the prospect that students might abandon the course entirely since it was an elective course and not tied to their curriculum. Moodle provided accurate time stamped data that gave us insight on which particular level students dropped the course from and drop-out was measured on a Dichotomous scale (0/1). If the Dropout Rate is 0, then the student has dropped the course at a certain level and in this case the student’s level of engagement will likewise decrease, and if the Dropout Rate is 1, that means that the student has successfully completed the courses’ mandatory parts (the 13 lessons). Exam results are used to compute the student's gained Knowledge, which is an indication of how much a student has learnt over the course.

Figure 3.2 illustrated above indicates one of gamification visualization implemented in the course. The road map was personalized and indicated each players’ progress in the course.

Fig. 3.2
An illustration depicts a stairway-shaped roadmap, numbered from 1 to 12 in ascending order. An alien-like figure is positioned at number 5.

Road map

Full size image

3.3.1 Participants and Procedures

The students from all around the world registered for this elective course, which implies that our sample size was not limited to BME students. The total number of participants in our research was N = 97. The number refers to the number of students who successfully completed the registration form and clicked the ‘Enrol me’ button on either of the two courses listed above. The course was promoted on the faculty webpage and on the faculty’s social media accounts on Facebook and Instagram. As previously stated, students were given the choice to choose from one of the two courses to complete. Students were informed about the various teaching approaches, but they were not informed of which course had been gamified until the end of the semester (Table 3.1).

Table 3.1 Courses participation
Full size table

The majority of students were between the ages of 19 and 24 and between the ages of 25 and 30. A large proportion of which were, as expected, university students interested in the course. Moreover 77% of our students were males and 16% were females. Most of our sample population reported working full time. Students were also questioned in regards of their digital skill sets. The Fig. 3.3 illustrates their answers.

Fig. 3.3
A horizontal bar graph illustrates students' digital skills. 55 students responded positively, indicating that they have their own computer or laptop, followed by the ability to remove and install software on their computers.

Students’ digital skills

Full size image

Almost all the students had their own computer or laptop, and almost 85% of them were able to instal software and update the operating system. Net promoter scores (NPS) were incorporated after the third lesson in both courses. The number of respondents in the gamified course was 19, however the number of respondents in the non-gamified course was limited to 11. Although the participation in this questionnaire was voluntary only one student which was enrolled in the gamified course skipped it. Most students’ overall happiness was on the upper half of the chart, and their average happiness score was 7.

3.4 Analysis and Results

3.4.1 Engagement Levels

We calculated the levels of engagement among students in both courses. Students who did not finish the course, i.e. dropped out at a given point in the course, had a substantially lower level of engagement. Comparing time spent learning on both courses, the results were significantly different. Completion data was the only relevant data in this measurement. The students enrolled in the gamified course have spent an average of 56.75 h (fifty-six hours and forty-five minutes) on the course which equates to two days, eight hours and 45 min. By contrast, the total duration of the video materials for the whole course is two hours and eight minutes. Analysing the non-gamified course data informed us that the average hours spent on the course is 22.7 h (twenty-two hours and forty-two minutes). The radar chart indicated below illustrates our research results transformed into percentages (Fig. 3.4).

Fig. 3.4
A radar chart compares student engagement levels in gamified and non-gamified courses. The drop-out rate in non-gamified courses reaches 80%, while in gamified courses, engagement and knowledge gained reach 50% and 60%, respectively.

Results radar chart

Full size image

To further verify our second hypothesis, we analysed our courses engagement levels data using SPSS. Firstly, we conducted Kolmogorov–Smirnov and Shapiro–Wilk normality tests to better address our sample distribution. Kolmogorov–Smirnov test indicated a value of 0.00 < 0.05, moreover Shapiro–Wilk normality test indicated a value of 0.03 < 0.05 and therefore our sample is not normally distributed. Consequently, Mann–Whitney U tests were conducted on our engagement metric (Pallant, 2016). Test results indicated a moderate to strong differences between students registered in courses A and B in terms of courses’ engagement levels. P. value: 0.033 < 0.05, and Mann–Whitney U value of 345.0. Mean rank of the non-gamified course group was also smaller than the mean rank of the gamified group 26.78 < 36.68. By carefully examining our findings, we can conclude the validity of our first hypothesis: H1. gamification improved students’ engagement levels. The optional chat module illustrated in both courses was not used by any of the courses’ participants.

3.4.2 Dropout Rate

If a student relinquished the course at any point and did not reach the final lesson/level, they were considered as ‘dropped out’. Analyses of drop-out rates in both courses the results were as follows:

  • Gamified course dropout rate: 75.8%

  • Non-gamified course dropout rate: 83.3%.

The previously reported statistics did not exhibit a significant difference between the two courses; however, it does indicate that students enrolled in the gamified course were less likely to drop out. Another intriguing observation was that several non-gamified course dropouts reregistered again for the gamified course, but none of the gamified course dropouts reregistered for the non-gamified course. In order to properly address our sample distribution, we performed Kolmogorov–Smirnov and Shapiro–Wilk normality tests on this variable as well. The tests indicated our dropout rate data is normally distributed 0.518 > 0.05 and 0.400 > 0.05 respectively. Therefore, we conducted an independent T test to validate our second hypothesis. Test results indicate a sig value of 0.008 < 0.05, t value of 7.631 and a degree of freedom df of 95 which empirically validates our second hypothesis H2, i.e. gamification decreased students’ dropout rates.

3.4.3 Academic Performance

Student progress has typically been measured via the use of standardized tests, and academic scores. We filtered students’ data, especially the ones that relinquished the course shortly after registering. Secondly, we examined Moodle logs and determined the points students have accumulated and averaged their results. The variation in the number of course participants had no influence on the outcome. The conclusion was determined by 11 revision exams placed at the end of each Lesson/Level, including the final examination. The average number of points obtained by students in the gamified course was 44.2, whereas this number was 37.98 in the non-gamified course.

We implemented the same research methodology on our third research variable (academic performance) and analysed our data on SPSS. After conducting Kolmogorov–Smirnov and Shapiro–Wilk normality tests, our findings indicated that our sample is not normally distributed 0.01 < 0.05 and 0.00 < 0.05, respectively. Similarly, to our engagement levels’ analysis, we performed Mann–Whitney U tests on our academic performance variable. Data analysis demonstrated a P value of 0.128 > 0.05, and Mann–Whitney U value of 390. The mean rank of the non-gamified course group was smaller than the mean rank of the gamified group 28.44 < 35.46. Based on our findings, we can infer that although our experimental methods and analyses revealed differences in averages between the two courses, that may be attributable to the gamification impact, our statistical data indicates that academic achievement is independent from gamification.

3.4.4 Satisfaction Levels

We evaluated students' satisfaction levels in both courses; however, we were unable to compare students' satisfaction levels across the two courses due to technical limitations. Students’ satisfaction was measured using a five points Likert type response scale where responses ranged between 1 (Strongly Disagree) and 5 (Strongly Agree). Questionnaire statements included: ‘The course did not meet my learning needs’, ‘I am satisfied with this online course’, ‘I would recommend this course to others’. Our students were overwhelmingly satisfied with the course, with 91% expressing their contentment in both courses. Approximately 90% of those who have enrolled in our courses have stated that they would recommend this course to others.

3.5 Discussion and Conclusion

Gamification educational advantages has been thoroughly addressed in previous literature; however, the pandemic presented the opportunity to empirically investigate those advantages in E-learning and distant education in maintaining students’ engagement levels and minimizing dropout rates. At BME GTK, we implemented a research analysis to evaluate gamification educational potential in a higher education context. Research results proved gamification advantages in influencing higher academic engagement and decreasing students’ dropout rates. In terms of performance levels, research findings revealed variations in students' mean test scores, although this conclusion was not statistically supported. The courses’ elective nature represented a hurdle in retaining students’ enrolment which was clearly featured in our dropout rates data. Students' commitment to the course has also been influenced by their genuine interest in the course content. Our research recommends implementing the same research methodology on a fundamental university course to reduce bias. The game features included in this research demonstrated their academic psychological influence. Our research also recommends informing the students of the gamified nature of the course which can increase academic competition; however, competition levels should be assessed as well in order to avoid creating extra stress that might jeopardize the overall implementation. The chat module implemented in both courses was not utilized by any student. Moodle’ chat module is archaic compared to newer communication tools; however, the lecturer did not utilize the chat feature as his primary method of communication. Using this feature might lead to a deeper sense of community in distance learning classes if it was initiated by the course lecturer. According to Moodle logs our interactive games and progress bar were clicked and visited more than 200 times. As a result, the mentioned components became one of the most frequently visited sections of the course. Despite the fact that the deployed interactive games had no major instructional value, they enhanced the students' learning experience and increased the amount of time they spent on the course overall. In order to improve the learning experience, it is beneficial to include a game that is related to the course subject. Even though Moodle only has a limited capacity to handle gamified components utilizing H5P at this time, it continues to be a viable option for universities throughout the world, particularly when compared to alternative learning systems that do not support gamified features. The instructional videos utilized in this course were pre-recorded, and students received no assistance other than what was included in their learning material. Time was also a significant technical disadvantage, as the entire course curriculum, including learning materials and videos, had to be created from scratch. This course was not limited to BME students and therefore we received and welcomed students from all over the world. Students had varying equipment, internet speeds and English literacy, which may have contributed to our higher dropout rates. There are likely to be additional significant differences between the two courses, which may be detected if the two groups of students were compared using a larger sample size.