Keywords

1 Introduction

A MOOC (Massive Open Online Course) can be defined as “an online course with the option of free and open registration, a publicly shared curriculum, and open-ended outcomes” [1]. These courses offer new opportunities to learn in a timeless and demographically unrestricted way. Their impact on higher education has been envisioned [2], reaching figures of more than 101 million enrolees up to 2018 [3].

Despite this remarkable success, one of the main problems of these courses is the funnel of participation [4] with a typical completion rate ranging between 5 and 10% [5]. MOOC instructors deal with a heterogeneous environment with many enrolees that differ in motivation, background and previous knowledge. Moreover, the high number of participants does not allow for personalised attention from the instructors.

Consequently, one important difficulty is that this new educational model requires learners to be autonomous due to the lack of personal mentorship. Participants are free to choose their own learning path, with little or no help to complete it successfully [6]. Thus, they need to self-regulate their learning to achieve their goals.

Nevertheless, directing one’s own learning process is complex, and many enrolees do not have the adequate self-regulated learning (SRL) skills. In addition, current MOOC platforms do not have enough mechanisms to support self-regulation [7]. Therefore, it seems essential to find complementary instruments which can assist MOOC learners and improve their SRL skills.

Despite the need to support learners’ SRL skills, researchers have mainly developed exploratory works to understand the characteristics of MOOC participants. Only a few research works implemented interventions to support SRL in MOOCs [8]. However, tools presented in the literature tend to focus on specific SRL sub-processes rather than accompanying the participant during all the learning process. In fact, authors in [9] highlight the importance for learners of receiving support during all the phases of SRL. Thus, there is a demand for more instruments to assist MOOC participants throughout their entire learning process.

In this line, we present an external tool aimed to help MOOC learners to develop and improve their SRL skills. This tool is intended to support learners in the establishment of their own learning path. Thus, this tool aims to help those participants who, wishing to do so, might not be able to complete the MOOC because they are not able to manage their learning adequately. This idea is concretised in the development of a Google Chrome extension or plug-in focused on three specific SRL strategies: (1) goal setting, (2) time management, and (3) self-evaluation.

2 Prior Work

2.1 Self-regulated Learning

In the field of educational psychology, self-regulated learning (SRL) is a major area of research, as it encompasses many influences of learning. One well-known definition of SRL [10] is Zimmerman’s [11]. He described SRL as learner’s generation of thoughts, feelings and behaviours to achieve his goals. One of his contributions is the Cyclical Phases Model, which considers self-regulation as a 3-phase cyclic process:

  1. 1.

    Forethought. In general, students plan their learning, including the personal objectives they want to achieve. Moreover, they reflect on their motivation to learn (e.g., to get a good grade, professional development, personal interest, etc.).

  2. 2.

    Performance. Among several skills, learners organise the necessary tasks and schedule some time to learn. Additionally, they monitor themselves to see how they are progressing. Thus, they can make changes if they are not satisfied.

  3. 3.

    Self-reflection. Students evaluate their learning performance and the reasons for it. After this reflection, learners decide what changes they need to make to adapt their strategies and improve these strategies in the next cycle.

2.2 Self-regulated Learning in MOOCs

Initially, self-regulated learning (SRL) was studied in traditional educational settings, where the student and teacher were in the same physical location and could establish personal contact. Later on, research on SRL has continued in online learning environments, such as MOOCs.

However, the differences between MOOCs and traditional educational settings create new challenges, mainly due to the heterogeneity and massiveness of the participants. Additionally, learners are expected to self-regulate their learning, although prior works have shown that many MOOC participants lack the needed skills. Therefore, there is a need for research works which carry out interventions to support self-regulation in MOOC participants.

Some authors have developed MOOC platforms to encourage learners to be active in their learning. For example, eLDa platform allows learners to choose the delivery mode between the instructor-led mode (with a pre-recommended order of lessons) and the self-directed mode (in which the learner freely decides which learning path to take) [12]. Moreover, elements to improve participants’ motivation and self-regulation are incorporated, such as lesson prerequisites, private messaging, forums, and progress maps [13]. In a pilot study conducted in 2015 [14], the participants of a programming MOOC in eLDa platform were offered a survey regarding their preferences in MOOCs. The results were that the participants mostly preferred short MOOCs, with short lecture videos, as they intended to spend less than an hour per day in the course. Finally, regarding the type of online course delivery, the answer distribution was: 15.5% preferred collaborative learning, 15.5% preferred instructor-led learning; 46% preferred interactive learning, and 23% preferred self-directed learning. Therefore, learners seem to prefer an adaptative approach, followed by the self-directed mode.

Other tools found in the literature to support SRL in MOOCs are focused on specific instructional elements, such as videos. This is the case of Video-Mapper, which is presented as a “video annotation tool” [15]. Learners can add annotations to specific time points in MOOC videos, as well as reading the annotations of the other enrolees. Thus, this tool aims to foster collaboration, discussion and interaction with the content of the MOOC videos. An evaluation of the tool with real MOOC participants showed its effectiveness and usability. Moreover, learners’ engagement is increased through the visualisations, as they can have an appealing overview of the lectures. Another constructive feature is the option to link questions to a specific time point in a video. However, it is difficult for a learner to keep track of his activities or his peers’. Video-Mapper helps learners to organise their learning path with the map structure of the lectures. Some SRL strategies this tool is intended to support are goal-setting and strategic planning. Enrolees can analyse what they have already done, and what they want to do according to their personal objectives. Moreover, the annotations are shared by all the participants, which helps to create a sense of community and fosters motivation and help-seeking.

Other authors focus on one SRL phase, such as NoteMyProgress [16]. This tool was developed to support the SRL strategies of the performance phase of enrolees in Coursera MOOCs. Specifically, the implemented and evaluated tool focuses on task strategies (note-taking) and time management. NoteMyProgress is composed of two elements: a plug-in for Chrome and a web application. The plug-in allows the learner to take notes while taking the course. The web application is a dashboard in which the user gets different personalised visualisations so that he can observe how he has interacted with the MOOC. For example, the user can get figures showing the number of videos watched and the time spent on them, or the percentage of time spent on the course versus the time spent procrastinating. After an evaluation with real users, positive results regarding the usability of the tool were obtained [16].

However, authors in [15] state that more effectiveness might be achieved by helping learners across all the phases of SRL. Therefore, we opted for designing a tool to support one SRL strategy per self-regulation phase, as modelled by Zimmerman’s SRL model [11].

3 Analysis

Before designing a tool to support SRL in MOOCs, we reviewed the literature to decide on which 3 SRL strategies this tool should focus. Regarding the forethought phase, goal setting is considered helpful in the literature [17, 18] but, in general, is a weak area for learners [17, 19]. Regarding the performance phase, time management appeared to be influential to succeed in MOOCs [20], although learners seem to have difficulties to effectively adapt their study time according to their goals [21]. Regarding the self-reflection phase, the self-evaluation strategy seems to have promising outcomes. For example, Learning Tracker is a tool which improved the performance of learners by fostering self-evaluation through social comparisons [22]. Additionally, emotions are an influential factor in learners’ interactions, whose regulation helps to predict the academic outcomes [23].

After this decision, the authors derived a set of requirements to support goal setting, time management and self-evaluation in MOOC environments. The summary of requirements is presented below:

  • Provide sign up and authentication for the users so that their information is not shared among other users. Data privacy is considered an essential requirement.

  • Provide the option of indicating the MOOC(s) in which the user is enrolled. The tool should have a list of supported MOOCs, from which the user can select those courses in which he is registered.

  • Support the creation of goals for the MOOC(s) in which the user is enrolled to foster goal setting. Additionally, the user should be able to view, mark as completed, and delete the goals he has set.

  • Support the notification of expired goals according to their due date. When a goal expires, an alert should be triggered to raise the awareness of the learner.

  • Provide a timeline display with the user goals according to the due date to foster time management. Specifically, the supported display should be a calendar with a colour code indicating the status of the goals which expire in each specific date.

  • Support the learner’s own evaluation of performance and emotions to foster selfevaluation. This self-evaluation should be simple to ease learners’ reflection. Providing several predefined options to choose from is advisable.

  • Provide some advices to the learners in order to improve their SRL skills as well as inspirational quotes to increase their motivation.

  • Provide an intuitive and appealing interface. Designing a simple and user-friendly interface is considered important to achieve high usability and learners’ engagement with the tool.

4 Design

4.1 Design Decisions

After determining the SRL strategies to support, the specific type of tool was chosen with the constraint of focusing on edX platform, as this was the institutional MOOC platform. In fact, other intervention types apart from tools were not suitable for our work. On the one hand, path recommendations or interventions that are embedded in the MOOC platform cannot be implemented in edX platform. On the other hand, study groups are not considered the best option in MOOC environments due to location constraints (e.g., Guo and Reinecke identified 196 origin countries among enrolees of 4 different MOOCs [24]).

Therefore, the initial designing step was selecting between a computer or mobilebased tool. Currently, the edX mobile application allows participants to watch videos and complete exercises. However, this application does not yet allow completing evaluations. Thus, enrolees who intend to complete the MOOC should access the course through a computer, at least, to take the exams. Additionally, a browser should be used when accessing the course. According to StatcounterFootnote 1, Google Chrome browser has a worldwide market share of more than 50% during 2017 and 2018. Consequently, the idea of a computer-based tool is concretised in the development of a Google Chrome extension (also called plug-in) to be designed according to the requirements summarised in the previous section. This tool, named MOOCnager, aims to support MOOC participants in their development of their SRL skills.

Finally, the last design decision was choosing a suitable database to store and access the information generated by the enrolees’ usage of the plug-in. A local database was discarded so that the information of a user was not constrained to the device he is accessing the plug-in with. In other words, the intended behaviour is that the information of a user is accessible, after logging in, regardless of the device. Additionally, a non-relational database (NoSQL) was preferred because possible future changes in the stored information do not imply major database modifications.

5 Implementation

HTML (HyperText Markup Language) and JavaScript has been used to complete the implementation of the plug-in, in addition to Firebase Realtime Database to store and retrieve the required information. The following subsections summarise the possible interactions between a registered user and the plug-in.

5.1 Enrolled MOOCs

The user can specify in which MOOC or MOOCs he is enrolled. This information is used so that the user is only able to set goals to his enrolled MOOCs. For the preliminary evaluation, MOOCnager is configured to include a provisional list of courses (specifically, 6 Java programming courses offered by UC3Mx), although it can easily be modified to add or remove courses.

5.2 Goal Setting

One of the supported SRL strategies, included in the forethought phase, is goal setting. Learners should set their personal objectives before their learning, to direct it towards achieving these goals. With MOOCnager, users can set goals for the supported MOOCs in which they are enrolled. To help participants with their goal setting strategy, some predefined goal types are included: finishing a specific unit, watching a number of videos, doing a number of assignments, or completing a number of evaluations. Each goal has an expiration date and belongs to one of two groups according to it: current goals (their due date has not arrived yet) and past goals (they have already expired). Figure 1a shows an example list of current goals. Users can remove a current goal or indicate that it has been completed. When a current goal expires, a notification appears so that the user indicates if it was achieved or not. Additionally, the list of past goals displays them with green font (if they were achieved) or red font (if they were not achieved).

Fig. 1.
figure 1

MOOCnager captions. From top to bottom, and left to right: (1a) Goal setting; (1b) time management; (1c) self-evaluation.

Full size image

5.3 Time Management

Time management, included in the performance phase, appears to be challenging to MOOC participants, according to prior literature. MOOCnager aims to support this strategy with a personalised calendar (see Fig. 1b). Each day is displayed according a colour code: grey, if no goals expire on that day; green, if all the goals that expire on that day have been achieved; red, if no goals that expire on that day have been achieved; and orange, if some of the goals that expire on that day have been achieved and some have not. Additionally, the goals that expire on a day are shown when hovering the mouse over that day. In this way, learners have a quick and general vision of their week and the goals they want to achieve. Thus, they might focus more on them and manage their time to accomplish their personal objectives.

5.4 Self-evaluation

The supported strategy of the self-reflection phase is self-evaluation. MOOCnager encourages users to reflect on their emotional state and performance by completing a form (see Fig. 1c). This form includes a first block to evaluate emotions and a second block to evaluate one’s own performance. To facilitate self-evaluation, the emotion and performance blocks have 6 predefined options each, among which the user can choose the one that best fits his evaluation.

5.5 Advices

Different advices, quotes or questions are included in MOOCnager to assist learners with their learning skills and motivation. There are different categories: goal setting, study place, concentration, time management, motivation, and other. For instance, a goal setting advice is “try to set SMART goals: S-Specific, M-Measurable, A-Attainable, R-Realistic, T-Time-Bounded”.

6 Evaluation

A preliminary evaluation has been conducted, conditioned by the brief available period of time to complete it. Learners of 6 Java MOOCs offered by UC3Mx were sent the link of Chrome Web Store corresponding to MOOCnager. However, the use of the plug-in was very low (only 38 users set a target, and 21 submitted a self-assessment). For this reason, a short survey was sent to the users to delve into the reasons for their low use, although only 5 users completed the survey. Regarding the main reasons for the low interaction with the plug-in, the participants could select a predefined answer or add a personalised answer. Results include not having enough time (2 respondents), not an attractive interface (1 respondent), prefer running a side project (1 respondent), and being inactive in the course (1 respondent).

Moreover, we also hypothesise that other factor influencing the low interaction with the plug-in might be that the MOOCs were very advanced when the plug-in was published. Therefore, only a small percentage of the enrolees were active. In fact, one survey respondent pointed this as his main reason for the low interaction. It is important to be aware that of those registered in the MOOC, only a small percentage of them access the course and are active during most of the course length [5].

In addition, the survey contained one free-response field whose answers showed that another important usability problem is that the tool is external to the MOOC platform. A user commented in the survey that he would have preferred a tool which is not external, as he claimed, “to forget that I even have a plug-in that tracks my progress”. Thus, a tool which is integrated in edX might have been more popular. However, this integration would require working with Open edX platform and not edX.

7 Conclusions and Future Work

MOOCnager aimed to help MOOC learners in their SRL, providing support during the 3 SRL phases [9]. The low use of the tool and the survey taken by some participants showed that learners have difficulties to manage their study time. In fact, this has been an important issue reason for participants to leave MOOCs [5]. Moreover, enrolees seem to prefer tools which are integrated in the MOOC platform and do not require any interaction from the user. Thus, we encourage collaboration between MOOC platforms and researchers to design tools that can give advices and support learners’ SRL based on their performance, without needing additional information.

According to our results, we propose revising the idea of the plug-in so that the tool is embedded in a MOOC platform, such as Open edX. However, although Open edX offers more flexibility for developers and instructors, MOOCs tend to be published in edX to obtain more enrolees. In the future, we will make the plug-in available to MOOC learners from the beginning, getting a more precise evaluation of the tool. We might reach the contradiction that those who most need support in their SRL are the least likely to use the MOOCnager spontaneously. Predictive analytics could be used in such a way that learners are shown that using the plug-in increases their chances of success in the MOOC, as a way of convincing them to use MOOCnager.

Moreover, results of research work in MOOCs might be constrained to the context (e.g., MOOC topic, length, delivery mode) due to the heterogeneity of the environment. Consequently, it might be interesting to study the plug-in usage among enrolees of other courses and analyse their similarities and differences.

In addition to MOOCnager, we implemented another system which uses the plug-in information (e.g., achieved and unachieved goals, self-evaluations) to create and send reports with personalised advice in order to help participants improve their SRL skills. As a future work, we propose the evaluation of this system with real users, as a complement of the plug-in. In this way, the intervention can be adapted to each learner and his performance, which seems to be valued among MOOC participants [25].