Keywords

1 Introduction

Distance educational systems have gained increasing use within institutions in the twenty-first century since they offer e-learning options to students and improve the quality of traditional courses in classrooms. These e-learning systems, such as Modular Object-Oriented Dynamic Learning Environment (Moodle), provide students different types of activities, such as preparation of assignments and quizzes, and engagement in discussions using chats and forums. Moodle is one of the most well-known free and open-source e-learning platforms which allows the development of interactive and simple online courses and experiences [1].

However, the distributed nature of distance learning has raised new challenges. For instance, unlike classrooms, it becomes much harder for teachers in distance learning to supervise, control and adjusts the learning process [2]. In massive open online courses, where thousands of students are learning, it is very difficult for a teacher to consider individual capabilities and preferences. In addition, the assessment of course outcomes in Learning Management Systems (LMSs) is a challenging and demanding task for both accreditation and faculty [1]. Anohina [3] stated that it is necessary to provide an intelligent system with adaptive abilities so it could effectively take the teacher role. Researchers suggested using Learning Analytics (LA) for representing important information about students online [2]. In this context, Siemens [4] defined LA as “the use of intelligent data, learner-produced data, and analysis models to discover information and social connections, and to predict and advise on learning”. Learning analytics is recently a hot topic among researchers and educators where various groups, societies, and journals are encouraging the research in LA field and the practice in higher education [1].

LA is often integrated into online learning environments, including Moodle, through the use of plugins. However, plugins usually require a considerable effort, most often involving programming, to adapt or deploy them [2]. This can limit their use by teachers. In addition, to the best of our knowledge, no plugin is reported online which provides real-time interventions to students for a better learning process. Additionally, several studies highlighted the effectiveness of applying gamification in online learning environments to motivate and engage students [5, 6]. Gamification refers to the use of game design elements, such as badges and points, in non-gaming contexts [7].

Therefore, this paper presents an intelligent gamified Moodle (iMoodle), based on a newly developed online LA system named Supervise Me in Moodle (SMiM), which: (1) provides dashboards for teachers to easily help them supervise their students online; (2) predicts at-risk students who might fail to pass their final exams. Specifically, the use of some game design elements might help in predicting students’ with lower performance and who can be at-risk of failing to pass their final exams; and, (3) provides real-time interventions, as notifications, by providing supportive learning content for students while learning.

The rest of the paper is structured as follows: Sect. 2 conducts a literature review about gamification and learning analytics. Section 3 presents the implemented framework of the gamified iMoodle with the use of SMiM system. Section 4 explains the experimental procedure for evaluating iMoodle and discusses the obtained results. Finally, Sect. 5 makes a conclusion with a summary of the findings, limitations and potential research directions.

2 Related Work

2.1 Gamification

Various approaches were proposed in the literature to motivate students and increase their learning outcomes. One of these approaches is gamification which refers to the use of the motivational power of digital games via the application of game design elements, such as badges and leaderboard, in non-gaming context to engage and motivate users [7]. According to Kapp [8], gamification is defined as “using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems”. Many researchers discussed the effectiveness of gamification in educational contexts [5, 9, 10]. For instance, Kim, Song, Lockee and Burton [5] stated that gamification is an effective instructional approach that is able to increase students’ motivation and engagement, enhance their learning performance and promote collaboration skills. Brewer et al. [11] also found that the application of gamification in a learning environment has helped in increasing the percentage of task completion from 73 to 97%.

Several game design elements were reported in the literature that can be integrated into educational contexts, but the most commonly used ones are Points, Badges and Leaderboards (PBL) [12]. In this context, Garcia et al. [13] investigated the efficiency of gamification by implementing PBL into programming course. They found that students’ performance in programming tests increased by using a gamified environment compared to a non-gamified environment. Similarly, an experiment study by Hew et al. [14] at an Asian university reported that the integration of points, badges and leaderboard have a positive impact on students’ motivation and engagement to involve more in difficult tasks. Barata et al. [15] also included game design elements like points, levels, leaderboard, challenges and badges to gamify a Master’s level college course and found that gamification can be an effective tool to enhance students’ attendance and participation,

Additionally, the implemented game design elements, such as points and progress bar, can also give an overview of students’ progress and performance in a given course. Therefore, several researchers suggested the use of these elements to motivate students and also to provide teachers with feedback about their students’ performance. This can further help them predict at-risk students [6, 16]. For example, the number of the collected badges from the submitted activities and students’ rank on the leaderboard, which is based on their collected number of points from their interactions with the learning environment, are indicators of students’ performance in the course, hence they can be used to help the system predict the students with low performance (at-risk of failing or dropping a class).

2.2 Learning Analytics in Moodle

Learning analytics has emerged as a very promising area with techniques to effectively use the data generated by students while learning to improve the learning process. Van Barneveld et al. [17] defined LA as “the use of analytic techniques to help target instructional, curricular, and support resources to support the achievement of specific learning goals”. Powell and MacNeill [18] identified five potential purposes of LA as follows: (1) provide students feedback about their learning progress compared to their colleagues; (2) predict at-risk students; (3) help teachers plan interventions when needed; (4) enhance the designed courses; and, (5) support decision making when it comes to administrative tasks.

Moodle offers several learning analytics tools to assess students’ performance and to help in evaluating different skills and competencies. For example, GISMO [19] is a visualization tool for Moodle which is used by teachers to analyze the learning process of all students. It is incorporated within Moodle as an additional block. It generates graphical representations to evaluate students’ behaviors, based on their log data. MOCLog [19] analyzes online students’ interactions and provides summative statistical reports for both students and teachers to enable them to better understand the educational process. Analytics and Recommendations [20] uses visualization techniques, namely colors and graphs, to provide information regarding students’ involvement in each activity of online course as well as recommendations to students so that they can improve their attainment. LAe-R [21] is a plugin which is based on the concept of assessment rubrics technique. LAe-R has various grading levels and criteria that are associated with students’ data identified from the analysis of their online interactions and learning behaviors. At-risk student reporting tool [22] provides information for teachers, based on a decision tree model, about students who might be at risk of failing a course.

All the above presented LA tools in Moodle focus mostly on offering various criteria which help teachers in assessing design aspects of the effectiveness of their provided online courses for improving their quality and for identifying opportunities for interventions and improvements. However, despite the fact that predicting at-risk students early in the semester can increase academic success [23], only one tool focuses on doing so (i.e., At-risk student reporting tool). In particular, this tool simply reports the at-risk students to teachers without providing them a medium for interventions to help these students. In addition, most of the above-presented tools are in the form of plugins which usually require a considerable effort, most often involving programming, to adapt or deploy them [2]. To overcome these difficulties, a new iMoodle is developed where its framework is described in the next section. iMoodle differs from Moodle by having a built-in LA system, namely SMiM, which easily helps teachers control the online learning process without going through the complicated process of installing different plugins to achieve different objectives (since every plugin has its own objective). iMoodle also differs from Moodle by providing students real-time interventions and support as notifications as well as predicting at-risk students.

3 Framework of the Intelligent Gamified Moodle (iMoodle)

Figure 1 presents the framework of the implemented gamified iMoodle [24]. iMoodle aims to predict at-risk students as well as model students’ personalities to provide them personalized interventions. Specifically, the student’s personality, as an individual difference, was considered in this research due to its importance and influence on the learning process and behaviors of students [25]. Therefore, modeling the students’ personalities, for instance, whether they are extrovert or introvert, can enhance their learning outcomes and specifically provide more appropriate interventions for them if they are at-risk [26]. However, this paper mainly focuses on predicting at-risk students, and personality modeling is beyond its scope. As shown in Fig. 1, during the learning process, the students’ traces are collected in an online database and automatically analyzed in order to extract knowledge and provide real-time interventions.

Fig. 1
figure 1

The developed iMoodle Framework

Full size image

A learning analytic system SMiM is developed and integrated into iMoodle in the Moodle block form where teachers can easily access it and keep track of their students in each enrolled course. SMiM has three layers, namely: (1) privacy layer keeps students’ traces safe; (2) analysis layer uses both data mining and visualization techniques to extract useful information for teachers; and, (3) reporting layer predicts at-risk students, implicitly model personality based on the students log data, and provides reports and real-time interventions while learning. Each of these layers as well as the gamified iMoodle are explained in the next subsequent sections.

3.1 Gamified iMoodle

To enhance students’ learning motivation and engagement, gamification was applied in our iMoodle. Specifically, to have an effective application of gamification, the self-determination theory was applied while designing our gamifed iMoodle. This theory is one of the motivational theories which is widely and successfully applied in gamified learning environments [13]. It is based on the fulfillment of students’ different psychological needs [27, 28], namely: (1) need for competence refers to the motivation to overcome challenges and achieved success. This can be satisfied using game design elements which provide feedback about students’ success to trigger the feeling of competence and challenge; (2) need for autonomy refers to self-direction and freedom of choices. This can be satisfied using game design elements which allow students to be in charge and make their own decisions; and, (3) need for social relatedness refers to the feeling of connectedness and being a part of a group. This can be satisfied using game design elements which can trigger the feeling of relatedness within students. Table 1 presents the selected and implemented game design elements in our iMoodle, their descriptions, and how they are related to the three psychological needs.

Table 1 Implemented Game design elements in the gamified iMoodle
Full size table

3.2 SMiM

The three main layers of the SMiM learning analytics system are detailed below.

Privacy Layer. This layer aims to keep the online students’ privacy safe with the login and password authentication method. In this context, to access the reports and information provided by SMiM, the teacher should have his/her session already active on iMoodle (i.e., the teacher has already entered his/her credentials to access iMoodle and chosen his/her courses). If not, the teacher will be redirected to the authentication interface. This keeps the information regarding students safe where only authorized teachers can have access to it. In particular, the student’s password is encrypted and stored within the online database. In addition, the Secure Sockets Layer (SSL) protocol is used to ensure a secured communication of students’ data within iMoodle. Furthermore, since the collected data and the obtained analytics results, recommendations and interventions should have a pre-defined time for how long they are going to be stored and used [29], the collected traces and generated reports are stored for a pre-defined period (one academic year) before they are automatically deleted.

Analysis Layer. This layer aims to analyze the students’ collected data in order to extract useful information for teachers, predict at-risk students and generate real-time interventions for them. Specifically, SMiM uses both data visualization and data mining techniques to analyze these traces. Data visualization is the use of computer-supported, interactive, visual representations of abstract data to amplify cognition. This can be achieved, for example, using tables, charts and histograms. In this context, SMiM uses data visualization to provide statistical reports for teachers to control the learning process and keep track of their students. Data mining, on the other hand, is the process of applying a computer-based methodology for discovering knowledge from data. In this context, SMiM uses association rules mining based on Apriori algorithm, to predict early in the semester at-risk students within iMoodle who would likely fail their final exams of a particular course, hence increase academic success by providing early support.

Association rule mining discovers relationships among attributes in databases, producing if-then statements concerning attribute-values. An \(X \Rightarrow Y\) association rule expresses a close correlation between items (attribute-value) in a database with values of support and confidence as survey by Shankar and Purosothmana [30]. In particular, Apriori Algorithm is used to find these association rules. It has two important variables: Minimum Support Threshold which is a support of an association pattern is the percentage of task-relevant data transaction for which the pattern is true (see equation a) and Minimum Confidence Threshold which is defined as the measure of certainty associated with each pattern (see equation b) [31].

$$\begin{aligned} ({\text{a}})\quad {\text{Support }}(X \Rightarrow Y) = \frac{{{\text{Number of tuples containing both }}X{\text{ and }}Y}}{\text{Total number of tuples}} \hfill \\ ({\text{b}})\quad {\text{Confidence (}}X \Rightarrow Y )= \frac{{{\text{Number of tuples containing both }}X {\text{and }}Y}}{{{\text{Number of tuples containing }}X}} \hfill \\ \end{aligned}$$

The Apriori algorithm developed within SMiM was first applied on previous learning dataset (knowledge base) from a public university in Tunisia which contains the final exam grades of students in a course and their learning behaviors within a classic Moodle. This was to extract the predictive association rules to detect at-risk students in iMoodle. In particular, based on a literature review, two types of factors are found that can help in predicting at-risk students namely, demographic and performance/behavior [32,33,34].

Demographic factors describe the students’ background and profile to identify the probability of students to successfully complete a course. However, since iMoodle aims to be used in both online and blended learning, demographic data would not work particularly well in this case because students can be from anywhere in the world. Performance/behavior factors, on the other hand, consider students’ actions in a course, such as what they viewed or submitted, as well as their performance on activities/assignments based on the assigned grades from the teacher.

Based on student performance/behavior, we selected five factors to help in at-risk students’ identification, namely: (1) Number of acquired badges which highlights the number of conducted learning activities, since every time a student finishes a learning activity, he/she gets a badge. This factor has been often used, for instance, by Billings [34], Xenos et al. [35] and Macfadyen and Dawson [36]; (2) Activities grades which refer to the value assigned by teachers to assignments and quizzes requested and delivered by students. In particular, if a student did not deliver an activity before its deadline, he/she receives a grade of zero. Also, if a teacher has not given the grade yet, this activity is not considered. In particular, the learning activities can be various assignments or quizzes that should be answered. This factor has been often used for designing early at-risk students’ warning systems, for example, by Macfadyen and Dawson [36] and Arnold and Pistilli [37]; (3) Student’s rank on the leaderboard which is based on the acquired number of points from his/her interaction with iMoodle (i.e., doing activities, participating in chat and forums, access to resources, etc.). For instance, if a student does not complete all the required activities and have low interaction with iMoodle, his/her score will be very low, hence he/she will be ranked at the bottom. Specifically, this factor presents an engagement trigger and an indicator of predicting at-risk students as highlighted by Liu et al. [38]; (4) Course progress which can be seen in the progress bar. It refers to the number of activities realized from the total of activities requested in a course. This factor has been recommended by Khalil and Ebner [16] to help in predicting at-risk students who have not completed the requested activities; and, (5) Forum and chat interactions which refer to students’ participation in online discussions, such as the number of posts read, posts created and replies. This factor has been often used by Liu et al. [38] and Khalil and Ebner [16].

Reporting Layer. After the analysis process is done (within the analysis layer), the reporting layer provides the generated reports and the automatic real-time interventions as follows:

Dashboard: SMiM provides dashboards within iMoodle for teachers to aid them control the learning process online and keep track of their students. This dashboard highlights the number of completion rate of each learning activity and quiz in each course, form, and chat interactions, the number of badges earned by each student, the progress of each student in the course and his/her rank on the leaderboard based on their collected number of points. For instance, as shown in Fig. 2, SMiM shows teachers the completion rate of each learning activity in the “Méthodologie de Conception Orientée Objet” (MCOO) course. This can help them keep track of their students’ progress online, hence not move to the next learning activity until they ensure that all their students have done the first one. Also, when the teacher clicks on each assignment, iMoodle shows the percent of students who got over and under the average grade. In particular, if students are at-risk, iMoodle provides real-time interventions, as notifications, by suggesting additional learning content support for them to further enhance their knowledge. The details regarding these provided supportive notifications are automatically stored in the database for future uses. Not only that, an interface is also shown for teachers where they can directly communicate with those students to help them pass the learning activities which they did not correctly finish.

Fig. 2
figure 2

Completion rate dashboard of learning activities within a given course

Full size image

At-risk students prediction: Through the use of predictive modeling techniques, it is possible to forecast students’ success in a course and identify those that are at-risk. Therefore, iMoodle, based on SMiM system, uses a predictive model (discussed in the analysis layer) as an early warning system to predict at-risk students in a course and inform the teacher. Teachers can then communicate with the at-risk students and provide them the required support for improving their performance in the course. Figure 3 presents examples of strong association rules obtained after running the Apriori algorithm. It is seen that the confidence of the association rules is very high (100%). In particular, the “forum and chat interactions” factor was excluded because over 75% of students did not use the forum and chat facilities. Finally, Fig. 4 presents the detected at-risk students based on the obtained association rules.

Fig. 3
figure 3

Examples of the obtained strong association rules

Full size image
Fig. 4
figure 4

Identified at-risk students in a given course

Full size image

4 Evaluation

An experiment was conducted to evaluate the technical reliability of the beta version of iMoodle. This experiment also evaluates the accuracy rate of iMoodle using SMiM in predicting at-risk students.

4.1 Experimental Design

The beta version of the iMoodle based on the built-in SMiM system was technically evaluated to test and enhance it if there were any bugs. In this context, the developed iMoodle was used for three months, in a public Tunisian university. The teacher was then requested to give a report highlighting the technical issues that were faced when using iMoodle. The feedback given by the teacher was then used to further work on the beta version and make it stable for future uses.

The post-fact technique was also used to mainly evaluate the accuracy of iMoodle in predicting at-risk students. This technique uses data from past events to understand a phenomenon. In this case, the data from a finished course on a classic Moodle was analyzed using the predictive model within iMoodle. The obtained at-risk students were then verified based on their exam grades to evaluate the accuracy rate.

4.2 Results

While the teacher reported that the developed iMoodle based on SMiM system helped her easily control the learning process and communicate with her students, several technical issues were found. For instance, the teacher reported that the automatic notification for students to provide additional supportive learning contents did not work for some learning activities. She also reported that some options within iMoodle (e.g., activate/deactivate notifications) should be disabled from the students’ learning sessions in order to not affect the learning process. These technical issues were fixed in our iMoodle stable version.

Table 2, on the other hand, presents the obtained results of the accuracy rate of predicting at-risk students within iMoodle. In particular, the number of correct results shows the number of students who are correctly identified within iMoodle in comparison with their final exams grades. The intervention layer within iMoodle, in this particular experiment, has no impact since the experiment is conducted using previous dataset and not from a current learning process. The efficiency of iMoodle in reducing the number of at-risk students is beyond the scope of this paper.

Table 2 Accuracy rate of predicting at-risk students within iMoodle
Full size table

As shown in Table 2, the accuracy rate of iMoodle in predicting at-risk students is almost 90%, which can be considered as sufficiently high. This means that our system is efficient in the prediction process. Particularly, only seven students were not correctly identified (i.e., they were at-risk but iMoodle identified them as not, and vice versa).

The obtained accuracy rate result was compared with other similar works, including the developed plugin for detecting at-risk students. For instance, Kotsiantis et al. [39] found that the accuracy rate of their system range between 63% and 83%. The prediction system of Da Silva et al. [22] had an accuracy of 85%. Liu et al. [38] and Khalil and Ebner [16], however, did not mention the accuracy rate of their systems in predicting at-risk students. To conclude, the developed gamifed iMoodle based on SMiM system has a better accuracy rate than the previous systems (which have mentioned their accuracy rates). Particularly, it can be deduced that the used factors, namely number of acquired badges, activities grades (in both assignments and quizzes), student’s rank on the leaderboard and course progress provide efficient combination for the at-risk identification.

It should be noted that it is very difficult to correctly identify all students since some students might alter their behaviors and put more effort to study outside of iMoodle (which cannot be detected) or fail the exam due to unforeseen events, such as becoming ill at the time of the exam.

5 Conclusion

This paper presented a new gamified and intelligent version of Moodle (iMoodle) which aims to help teachers control the learning process online and keep track of their students. iMoodle provides, based on a built-in LA system called SMiM, a dashboard for teachers to help them understand the learning process and make decisions. It also provides an early warning system by detecting at-risk students, based on various factors extracted from the literature, using association rules mining. Finally, iMoodle provides automatic personalized supportive learning content as notifications for students based on their behaviors online. The beta version of iMoodle was tested for three months during the first semester and several technical issues were identified and fixed. Furthermore, the predictive model was evaluated and the obtained results highlighted that iMoodle has a high accuracy rate in predicting at-risk students.

Despite the promising results, there were some limitations of the experiment which should be acknowledged and further investigated. For instance, the effectiveness of the iMoodle in learning was not evaluated. Also, the detection process of at-risk students was from only one course which has limited number of students (only 61 students). Future research work could focus on: (1) using the iMoodle and compare its impact on learning outcomes and technology acceptance with a classic Moodle; (2) investigating the efficiency of iMoodle using the intervention layer in reducing the number of at-risk students and increasing academic success, in comparison with a classic Moodle; and, (3) further develop iMoodle to provide as well personalized interventions based on students’ personalities.