Keywords

1 Introduction

Today’s online learning has offered rich educational resources in multiple media, including social media and the capability to support synchronous and asynchronous communication between instructors and learners as well as different types of learners, with the advent of Internet and World Wide Web (WWW). Hence, the potential for approaching to learners from the worldwide has been risen progressively, and the adoption of online learning or electronic learning (e-learning) from the higher education institution (HEI) and corporate training institution also increased rapidly. Globalized online learning is the ninth shift of Malaysian Education Blueprint, which has highlighted by Ministry of Education, Malaysia, for the transformation of Malaysian higher education system (Ministry of Education, 2015). Massive open online course (MOOC) is one of the main strategies that were initiated by Malaysian Government, with the Malaysian Education Blueprint for higher education, National Economic Model, and Economic Transformation Program (Mansor, Latifah, & Tengku Amina, 2015; Nor Fadzleen, Rose, & Naoki, 2015). MOOC has emerged as a significant environment for online learning, yet little is known about how user actually learns in a MOOC (Malligan, Littlejohn, & Hood, 2016).

MOOC is an online course which consists of recorded online video lectures on any given topic by one or more university professors and educators that is then easily accessed online by anyone around the world (Goh, Kaur, & Chion, 2015). Cloud system is hosted all learning materials, and the learning contents are delivered online so that it can be accessible from anywhere on the Web. The first Malaysian MOOC is launched by a private higher education, named Taylor’s University, which offered entrepreneurship by the Dean of School of Engineering via OpenLearning platform (Mansor et al., 2015; Nor Fadzleen et al., 2015; Muhstak, 2014). The pilot MOOC has initially created with the goal to promote higher education institution’s presence online learning and designed to attract students to further their studies.

Moreover, Malaysia also became the first country in the world to implement MOOC initiative for public universities (Nor Fadzleen et al., 2015; Ratnaria, Azizudin, Bakri, Muhammad, & Khaliza, 2015). The objective of this initiative is to boost the ranking of Malaysian higher education on global scale, by providing globalized, opened online learning experience without the limitation of time and accessibility (Nor Fadzleen et al., 2015). In Malaysia, the MOOC movement is still new and most of the HEI has just begun to catch on its wave and making MOOC to fit into their e-learning or blended learning system and platforms. The adoption of MOOC is also a strategy to bolster the international visibility and the university’s reputation (Ayub & Lim, 2016). With the emergence and implementation of MOOC in HEI, there are same issues regarding the concept of MOOC, questioning about pedagogical elements in MOOC (Gene, Nunez, & Blanco, 2014), assessment system and interaction between instructors and learners, and low completion rate (Chen & Chen, 2015; Gene et al., 2014).

Thus, the purpose of this study is to measure the effectiveness of courses delivered using MOOCs in Taylor’s University. The Kirkpatrick’s four levels of evaluation model have been adopted to evaluate the effectiveness of the MOOCs. The results presented in the following section relate to the identified research questions. The research questions are as follows: (1) How effective are the courses delivered using MOOCs and (2) what are the issues faced when using MOOCs? Although MOOC is the peak of the hype cycle of the education system in year 2013, it will be gone down, at the midst of the “trough of disillusionment.” Hence, this study has yet to represent MOOCs’ effectiveness in all higher education system; however, this survey study has explored on how effective the MOOCs that conducted in a selected higher education institution. The researchers hope the findings from this study will be able to contribute to the community of MOOC and other HEI that plan to implement MOOCs.

2 Literature Review

As a disruptive innovation, massive open online course (MOOC) has brought to revolution in education and made potential changes in the existing higher education landscape. Availability, affordability, and opportunity are three features of MOOC, which provides high quality of learning contents to the worldwide students. MOOC is used to promote lifelong learning which is crucial to increase students’ knowledge, deepened thinking skills, improve attitudes, and achieve learning goals in future life. Thus, MOOC provides free and open education to students from the world and is currently leading to new learning scenarios (Munoz-Merino, Ruiperez-Valiente, Alario-Hoyos, Perez-Sanagustin, & Kloos, 2015).

MOOC has shown a convergence of interests in social, economic, and technology developments in education in a global context. Five changes have been explored in higher education landscape (Yuan & Powell, 2013): (1) the expanded momentum of globalized learning; (2) the increased demand for accessibility to higher education; (3) the changing of learners’ demographics, experiences, and the growing numbers of lifelong learners; (4) the increased accessibility of social media and promote personalized learning (Grosseck, Holotescu, Bran, & Ivanova, 2015; Yuan & Powell, 2013); and (5) the changing model for higher education cost and affordability. Moreover, MOOCs also provide HEI with a platform to think creatively and innovatively and to explore new pedagogical practices, business models, and flexible learning paths in their provision. Hence, openness, a vital feature in driving educational innovation and transforming higher education system in the world (Yuan & Powell, 2013 ).

The great deal of MOOC is they will provide free access and cutting-edge courses which can drive down the cost of university-level education and potentially disrupt the existing models in higher education. Many platforms are used to provide MOOCs in HEI, such as edX, Coursera, FutureLearn, and Audacity (Yuan & Powell, 2013). Some studies of the MOOC believe that MOOC can benefit the HEI, lecturers, and students. For instance, MOOCs represent the ultimate democratization of education, by offering the easy accessibility to all. In most cases, participants subscribe MOOCs for free of charge and in some cases for a small or minimal fee to obtain a completion certificate (Hew & Cheung, 2014).

Nevertheless, the effectiveness issues of MOOC need to be explored to ensure MOOC really can be implemented in Malaysia in future. Strategic discussion concerning the disruptive potential of MOOC in higher education institutions needs to be conducted. Thus, higher education institutions need to make decisions about how to serve their specific mission and how to react to the different needs of students in a rapidly changing educational market.

3 Methodology of the Study

Each of the four levels of the Kirkpatrick model was applied in this survey. This paper will objectively discuss the effectiveness of Taylor’s MOOCs based on the Kirkpatrick’s model adopting quantitative and qualitative approach. This model helps to objectively analyze the effectiveness and impact of training, and this method was also used to analyze the MOOCs as it is important for the course creators and instructors to get feedback on how the MOOCs have impacted the learners and how they may improve the courses in the future.

The measures in Kirkpatrick’s model are recommended to preliminarily evaluate Taylor’s pilot MOOCs to get an all-encompassing evaluation of learning using MOOC as pedagogy. The following are the measures adopted into the survey given to the students after they have completed their MOOCs:

  • Level 1: Learner’s reaction—what they thought and felt about the MOOCs and whether the learning experience met their expectations (the Learning Environment)

  • Level 2: Learner’s learning—what they thought and felt about the learning environment that is OpenLearning in MOOCs and whether they thought the MOOCs met their expectation and increased their ease of obtaining knowledge or capability (Overall)

  • Level 3: Learner’s behavior—in what they think the MOOC is relevant and helpful for their future success also the extent of behavior and capability improvement and the application is also looked for (Relevance)

  • Level 4: Effectiveness results—the effects of MOOC on student’s learning and their confidence on applying what they have learned as a result from taking up the MOOC and the engagement from facilitators and their peers were also observed qualitatively (Energy for Change)

This study is a descriptive study. This study is used to collect different types of information and description from different perspectives to determine the effectiveness of MOOC in Taylor’s University via a survey research method. Data were collected via a questionnaire when the participants had completed their MOOCs. Taylor’s University is estimated having 4000 students in population. As mentioned above, there are 250 students be the respondents for this study. Thus, the sample proportion of the population with the population size = 4000 are equal to 0.0625. Since the sample proportion value is more than 0.05 (5%), therefore, the number of subjects is sufficient for the study (Chua, 2012).

4 Findings and Discussion

The findings of the survey will be presented and discussed based on two research questions as mentioned in the introduction, the effectiveness of MOOCs and the issues of using MOOCs from the learners’ perceptions.

4.1 The Effectiveness of MOOCs

  1. (a)

    Helpful study materials and suitable program

One of the major findings for the effectiveness of MOOCs is that the learners find the program is suitable to a wide range of learners. The curriculum in MOOCs featured a publicly shared curriculum with open-ended learning outcomes. According to the learners, the content-based MOOCs are anchored in content. It is free access to all learning materials such as lecture, small videos clips, text-based readings, slides, practical exercises, audio files, URL to other resources, online articles, discussion forums, case study, self-reflection, and peer feedback.

The learners expressed that all the contents in MOOCs appeared in a structured form with video and appropriate assessment timeline. It makes study more accessible, affordable, and attainable. The lecture slides and learning materials are helpful. The use of short videos is interconnected with formative quizzes that are based on lecture slides. The exercise instructions are clear and easy to understand. It helps the learners in their quizzes and assessment.

The finding shows that the learners like the downloaded lecture slides. The learners do not have to spend time on compiling study materials. All the finished works are saved automatically. The learners attempt to model a real-life project after completed the MOOC. Hence, effective materials can encourage and sustain engagement and thereby assisting learning. Curriculum activities, course activities, assignments, and assessment need to be considered in order to design an effective MOOCs. Assessments are encouraged to be built around a mapping between intended learning outcomes and objectives of the topic (Terras & Ramsay, 2015).

  1. (b)

    MOOC is useful for future reference

The learners think that MOOCs can greatly improve their knowledge for their future working environment. According to the learners, learning using MOOCs has given them the opportunities to improve and update their portfolio. The certification of the course is valuable to be added into the learner’s CV. One of the learners expressed that videos and activities in MOOCs related to field trip and analysis have created the inspiration and ideas for them especially in their studio design.

When the learners read the articles posted in MOOCs, the learning materials have created their curiosity and they start to research further for more knowledge. They have a clearer idea on what they are learning and how it works. The learners admit that the knowledge that they have learned in MOOCs is benefit to them in the future as they have a better understanding on the topics. This provides them the opportunity for better income and continues learning throughout life.

Gomez-Zermeno and Garza (2016) also agreed that MOOC is an alternative educational method for professionals who look for complimentary training and education. MOOC is suitable for people with professional education and experience to expand their knowledge (Ari, 2013). Furthermore, the learners highlighted that they started to apply what they have learned in MOOCs to their real life. They agree that the course allows the acquisition of new knowledge and skills in their field. It has widened their knowledge after completed the course. They mentioned the relevance of MOOCs to the subject area they are studying. The learners are able to model a real-life project after the tutorial in MOOCs. They started to apply what they have learned in MOOCs to their real life. Table 29.1 has presented the learners’ thoughts about the MOOCs for future reference.

Table 29.1 Learners’ think that MOOCs are useful for future reference
Full size table
  1. (c)

    Help learners to understand a certain topic better

We can see from the result that the majority of the learners engaged in the collaborative group discussion. Learners think that MOOCs are good platform to voice out and share opinions. All the learners have the chance to answer the questions and share opinions because there might not be enough time for everyone to answer the questions in class. With the flexible and adaptable means of delivering content, learners are able to learn and share knowledge with their peers through discussion forum. In MOOCs, each topic is related to an issue and the learners are required to express their opinions and complete the activities after reading each topic. The step-by-step course helps the learners to grasp the topics. This way helps them to understand the topics better.

  1. (d)

    Strongly recommend the course to other people

By analyzing the data, we can see that learners are quite satisfied with the courses provided using MOOCs. They generally feel that MOOCs can raise their interest in learning and learning efficiency. This is mainly because students in information age have a great sense of curiosity and the ability to adapt to the MOOC environment (Ji, 2016). From the survey, the learners share their opinion that they will strongly recommend the course to people that they know because it is interesting and well-organized. They agreed that MOOCs have enhanced their knowledge. Table 29.2 also presented the recommendations from the learners’ perceptions.

Table 29.2 Recommendations from the learners’ perceptions
Full size table
  1. (e)

    The program has met the learner’s expectation

The program has met the learner’s expectation. They think that it is fun besides gaining extra knowledge and experiences. The program is well-organized with precise and clear instructions. Furthermore, it is simple, comprehensive, and free. The learners like the way where all the information can be shared among the learners. The learners can do some research before responding to the questions. There is a two-way communication in MOOCs. They can discuss and share their opinions with their peers in the discussion forum. MOOCs have provided comprehensive insights to the learners on certain topics. MOOC teaching platform gained certain effects and inspired students’ interest in learning and improved their learning efficiency. The findings from (Garza, Sancho-Vinuesa, & Gomez Zermeno, 2015) also mentioned that 43.3% of the participants have met their expectation where MOOC helped them to acquire enriched learning. 23.6% of the participants have experienced higher learning proficiency compared to a classroom environment.

  1. (f)

    Information is sufficient for learners to apply in the future as a basic course, but more detailed information is needed for advanced course

According to the learners, the learning materials are sufficient for beginners in order for them to utilize in the future. It is useful for those learners who have less experience in the topic. The learners have received various resources from the module. It works as a starting point for them to further reading and learning about it. Nevertheless, the information may not be sufficient for those seeking for an advanced course. More details are needed for those who would like to learn more. Practices and activities are still required to make the learners more familiar with the topics. Current MOOCs help to spark the interest of those who are curious about the topics. They can learn some topics which they cannot learn in the classrooms. The learning process helps the learners to know when and where they can apply the knowledge. It also provides revision for some basic topics. Table 29.3 has shown the learners’ perceptions regarding the sufficiency of information in MOOCs.

Table 29.3 Learners’ perceptions regard to the sufficiency of information in MOOCs
Full size table

4.2 The Issues of Using MOOCs

Some issues are identified from the survey, which are including lack of knowledge in using MOOCs for novice learners and lack of support and guidance from the facilitators. Most of the learners are fresh and new to MOOCs. It takes some time for them to know how to start using MOOCs because some of the learners have limited IT knowledge. Some of them do not know how to begin to use MOOCs as a learning tool. They have difficulties with the structures of the course and lack of tutorial to guide them.

They would like to have some technical support and time management to plan for their course activities and assessment. Digital literacy skill of MOOCs is one of the factors to engage learners successfully. It is crucial to have an insight into how learners interact with technology in MOOCs in order to provide necessary training and guidance to the learners. Support for diverse MOOC users is essential for this new learning paradigm to enable learners to adapt to this new learning method. Workshop can be conducted to educate the learners on how to use MOOC. Help and support are important for the learners especially those who use it for the first time. Good support from various teams will help the learners to use the MOOCs easily. This includes welcome tutorial to guide the novice learners on the function and structure of MOOC.

Besides having the necessary knowledge and skills on how to use MOOCs, the learners would like to have a better understanding on the requirement from their facilitators. Some of them do not sure whether the answer that they have provided has fulfilled the requirement of the assessment criteria. They were afraid that the answer that they provided is wrong. (Terras & Ramsay, 2015) Hence, individualized feedback is one of the most important ways of keeping students engaged in their studies.

Furthermore, most of the learners are working adults. Level of involvement in MOOC may vary. There is a time constraint for them to do the activities in MOOCs. The workload in their career is high. They have less time to do the assessment and submit on time. Hence, it is important for the learners to define their own learning paths and not navigate the content and assessment tasks in a linear manner (Rolfe, 2015).

5 Conclusion

In conclusion, the learners have perceived that MOOC is an effective course with helpful study materials. It is useful for their future reference and allows them to apply what they have learned in real-life experience. The program has met the learners’ expectation, and they will strongly recommend the course to other people. The findings have shown that participating learners’ perceived level of MOOC satisfaction was generally at a high level. MOOCs make the teaching more effective by reforming the traditional learning methods. Nevertheless, the learning materials are sufficient for learners to apply in the future as a basic course, but more detailed learning materials are needed for advanced course. Issues such as the structure of MOOCs, support, guidance, and motivation are crucial for successful learning. The lack of skills, motivation, and support is the contributory factor for lack of engagement in MOOC (Garza et al., 2015). Terras and Ramsay (2015) also agreed that digital literacy skills, individual motivation, and self-regulation are the key learner attributes to maximize the learning potentials of MOOC. This study provides some insights into the effectiveness of MOOC and issues faced. Course designers should take the issues into consideration when designing MOOCs.