1 Introduction

Learning management systems (LMS), also called digital learning environments, online learning environments, course management systems or virtual learning environments, are web-based platforms that enable teachers to create online courses. LMS are an important issue in the research field of web-based instruction (Badia 2015), as they are widely adopted by schools and universities (De Smet et al. 2012; Pynoo et al. 2011).

Moodle is one of the most commonly used LMS for developing online academic courses, and several recent contributions have emphasized the significant influence of university teachers’ and students’ perceived usefulness of the Moodle platform in its use as a whole. It has been demonstrated that perceived ease of use, positive attitude, and perceived usefulness are the key factors in determining Moodle usage by students (Arteaga and Duarte 2010), and by students and teachers (Escobar-Rodriguez and Monge-Lozano 2012). Additional research has also demonstrated that communication has a strong effect on students’ perceived Moodle performance (McArdle and Bertolotto 2012). Five critical factors (which include content completeness, content currency, ease of navigation, ease of access, and course staff responsiveness) affect students’ satisfaction (Naveh et al. 2012), and satisfaction has shown to have a significant effect on behavioural intention to use Moodle (Damnjanovic et al. 2015).

Whereas the relationship between Moodle’s perceived usefulness and acceptance, and the Moodle platform use as a whole has been widely acknowledged at a university level (Pynoo et al. 2011), little is known about how this significant relationship works in secondary education. We consider that any study that seeks to provide significant information to be easily utilized by other researchers or practitioners, in terms of teaching and learning, should use conceptual categories directly related to the education field. Consequently, the purpose of this study is to explore in depth secondary teachers’ perceptions of the learning impact of Moodle usage by studying the links between the use of 12 Moodle activities and a set of Moodle learning impacts.

2 Literature Review

2.1 Types and Uses of Moodle Activities

Social constructionism theory underpins the technological design of Moodle. As a learning-centered management system, learning is considered a process of constructing knowledge by negotiating meaning with others and creating shared cultural artifacts. This theoretical assumption is put into educational practice by means of two sets of tools: resources and activities. Whereas resources include digital content files, activities make the learning task central and contain tools for discussion, sharing ideas and engaging in the construction of knowledge (Cole and Foster 2007).

Correspondingly, activities should not be seen merely as modules, tools or components of Moodle, but rather a set of different learning-tasks supported by a specific technological resource. According to Blin and Munro (2008, p. 483), activities “provide affordances for learner–learner or teacher–learner interaction as well as for manipulation and transformation of content”. From the point of view of the teacher, selecting an activity means promoting a certain type of educational interaction and a specific method to learn the content.

Moodle includes 15 types of activities that can be classified among seven main categories: creation and sharing of a collection of data (database); organization of a set of instructional materials (lessons); delivery, collection, evaluation, and feedback about content (assignments, workshops); communication and exchange of ideas (chats, forums, news); collaboration by building shared knowledge (glossary, wikis); assessment of student learning (choice, quiz, survey, and feedback); and reusability of learning resources (SCORM, and external tools) (Costa et al. 2012; Piotrowski 2010).

Very little research has been conducted about the use of Moodle activities in formal courses, and all contributions reviewed were carried out in universities. The analysis of data extracted from the Moodle database in a large amount of university courses showed that the most used learning activities were forum (78%), assignment (10%) and quiz (5%). Collaborative and reflective activities such as wikis (1%) remain marginal (Blin and Munro 2008). A complementary study which also used data from seventy virtual Moodle-based university classes demonstrated that the number of log entries related to informational interaction level, which focuses mainly on the reception or posting of content without feedback, were higher than log entries related to the communication-interaction level, which focuses mainly on the content exchange through communication or transactional interaction (Hamuy and Galaz 2010). When data was extracted by means of a questionnaire applied to university students, findings showed that the activities with a higher number of users were assignment (59.20%), forum (48.41%), and questionnaire (45.95%), and those with fewer users were chats (16.36%), and quiz/survey (17.86%) (Costa et al. 2012).

2.2 Technology Learning Impact in Classrooms

The concept of technology learning impact from the point of view of teachers is defined as the teachers’ beliefs about how technology improves teaching and learning (Petko 2012). In some occasions, it has also been called instructional benefits of technology, and it is part of a more general concept known as teachers’ beliefs regarding technology (Inan and Lowther 2010). Nowadays, there is not exactly a clear overview of how technology impacts learning in the classroom, due to the relation between the use of technology in the classroom and its instructional benefits for teaching and learning having merely been outlined (Voogt et al. 2013).

Several studies not strictly focused on the use of Moodle consider that the instructional benefits of technology include increasing the level of student motivation and helping students to achieve better text writing skills (Van Braak et al. 2004). Some theoretical contributions suggest that the positive impacts of digital technologies in the classroom the introduction of new teaching processes and the enhancement of student performance, which includes the development of digital competency and academic performance in basic subjects (Bilbao-Osorio and Pedró 2009). More recent studies include new learning benefits related to collaborative work, learning outcomes, learning interest and creativity, and learning strategies for the students (Petko 2012). Finally, some new technology learning impacts have been added to the list, such as the access to a wider range of learning content and resources, and the possibility that students become more motivated, attentive, active and independent in their learning process (Perrotta 2013).

Two recent studies have identified a set of more organized teachers’ perceived technology learning impacts in schools. Badia et al. (2014) identify the following instructional benefits of technology on teaching practices: the usefulness of technology-based communication and collaboration with students, the availability of educational resources on the internet, the improvement of the quality of students’ learning experience, the achievement of the educational goals, and the suitability of technology-based educational resources. In addition, Gómez and Badia (2016) identified five potential instructional benefits related to the use of mobile learning in primary education from the teachers’ perspective, which include providing new ways to learn, increasing learning engagement, fomenting autonomous learning, facilitating access to information, and promoting collaborative learning.

Less information is available about how LMS platforms impact learning in classrooms. In a traditional university classroom, whereby Sakai LMS has been used, teachers stated that efficient communication using content sharing, announcements, assignments, and syllabus tools, were valued more highly than social interactive aspects of Sakai related to the use of chat, discussion, and wiki tools (Lonn and Teasley 2009). Also at the university level but in an online course, six instructional benefits of Moodle as a teaching tool in Physics were roughly outlined: improving the organization, management, and delivery of course materials; providing a great number of resources; allowing the creation of attractive learning activities; making the teacher–students interaction easier; allowing them to share knowledge and difficulties among students; and reinforcing student abilities and knowledge (Martín-Blas and Serrano-Fernández 2009). Finally, university students stated that the use of Moodle had a positive effect on their perceptions of the quality of learning, the quality of information present in the learning materials, the learning performance and outcomes, and the intention to use Moodle in the future (Damnjanovic et al. 2015).

In summary, bearing in mind the aforementioned studies, both related to technology and LMS platforms, we conclude that several different kinds of impacts of technology in learning may be identified: enabling the sharing of knowledge and difficulties among students, improving the quality of course content in providing a greater number of resources, promoting the quality of learning and allowing the creation of attractive learning activities, promoting efficient communication and teacher–student interactions, and reinforcing student abilities, knowledge building, and learning performance and outcomes.

Taking into account the lack of knowledge about Moodle activities used in secondary education, we have considered it relevant to carry out an empirical study focused on several issues related with this field of study. Bearing in mind the available literature and studies, we attempted to answer the following research questions:

  • RQ1: What is the frequency of use of Moodle activities in the secondary classrooms?

  • RQ2: Do the teachers perceive that some relevant factors about teaching and learning in classrooms can be affected by the usage of Moodle?

  • RQ3: What are the differences in the perceived learning impact of Moodle activities among users/non-users of each activity?

3 Research Design

3.1 Context of Study

This research is based on data collected from 43 secondary schools which has received considerable support from the Catalan Ministry of Education since 2007 to make the technological integration and educational use of Moodle by teachers and students in the classroom possible. All secondary schools maintained a suitable technology infrastructure to run Moodle properly, consisting of complete Internet access anywhere via Wi-Fi, access to Moodle for all members, at least 63 computers per 100 teachers, 28 computers per 100 pupils, and 21 computers with an Internet connection per 100 pupils, and technical and pedagogical human support for the teachers and students in their respective schools.

Teachers of all centers have access to at least twelve Moodle activities (Cole and Foster 2007): (1) Database, which enables participants to develop, display and search a bank of record entries on any conceivable topic; (2) Chat, a synchronous conversation tool; (3) Choice, which enables teachers to ask a single question and offer a selection of possible responses; (4) Quiz, which allows the development of a set of questions of various types, including multiple choice, true–false, and short answer questions; (5) Survey, which provides a number of verified survey instruments; (6) Forum, which contains asynchronous messages for all site participants to read or respond to; (7) Glossary, which enables participants to create and maintain a list of definitions; (8) External Tools, which are activities and learning resources on other websites with LTI (learning tools interoperability) support; (9) Lesson, which allows teachers to create a linear set of content pages or instructional activities that offer a variety of paths or options for the learner; (10) Workshop, which enables the collection, review and peer assessment of students’ work; (11) Assignment, which allows students to upload and submit assignments and projects, and allows instructors to grade and comment on students’ submissions; and (12) Wiki, a collaborative document writing tool.

3.2 Participants

We made contact with principals of 102 secondary schools to seek secondary teachers who complied with three requirements: (a) They have attended specific training courses of Moodle activities during the last 5 years; (b) Teachers had used Moodle regularly throughout the year 2014; and (c) Over the year 2014 Moodle ran properly in the school and teachers received sufficient technical and pedagogical support when required. At the end, 132 teachers from 43 secondary schools fulfilled a questionnaire of this study (42.16% of response rate); this questionnaire gave information about the use of Moodle activities of 2850 pupils. The complete information of participants is shown in Table 1.

Table 1 Participants’ individual information (N = 132)
Full size table

3.3 Data Collection

An online survey was conducted during the last 3 months of 2014. In the first step, researchers communicated with principals and heads of departments asking to provide contact details of the teaching staff who fulfilled the requirements. Then an e-mail was sent to participants with the link to an online survey, which was developed using GoogleForms. The research has been developed in accordance with the ethical standards of the American Psychological Association (2010).

The online survey contains three sections. The first section (24 items) requested the socio-professional background (7 items), training experience with Moodle (2 items), and technological access conditions (15 items) of the teachers. Items regarding the technological access conditions were assessed using a 5-point Likert scale from 1 = “strongly disagree” to 5 = “strongly agree”. The second section measured the frequency of use of Moodle activities using the following sentence: “Assess how you or your students have used the following Moodle activities at this educational level selected.” It was assessed using 12 items, one for each Moodle activity, which ranged from 1 = “Never used” to 5 = “Used every 2 or 3 days”. The third section measured Moodle’s learning impacts perceived by teachers, and encompassed 21 items (see “Appendix”). Items ranged from 1 = “strongly disagree” to 5 = “strongly agree”.

The development of the three sections of the survey was based on aforementioned literature, and had been done taking into account previous studies also related to instructional benefits of technology (Badia et al. 2014; Gómez and Badia 2016).

3.4 Data Analysis

Data were analyzed using IBM SPSS Statistics for Windows, Version 21.0. There were three consecutive steps for the data analysis.

In the first step, descriptive statistics of Moodle activities use were calculated and classified according to the categories suggested by Piotrowski (2010) and Costa et al. (2012). As a second step in the data analysis, an exploratory factor analysis of the survey’s items related to the teachers’ perceived learning impact was carried out using the maximum likelihood extraction method. To reduce item variability to a multidimensional semantic space representing teachers’ meanings, five factors were identified for Moodle’s learning impact. The raw scores were added and divided by the number of items included in each factor to retain the original scale (i.e., ranging from 1 to 5, to facilitate its interpretation). In the third step, a set of analyses of Student’s t test and, when required, Mann–Whitney U nonparametric test, were conducted to examine the differences between teachers’ users and non-users of each Moodle activity, regarding all perceived learning impacts. The effect sizes of both statistical tests are reported. For the case of t tests, Cohen’s d has been used, and for Mann–Whitney tests, eta-squared has been used. The group of users was delimited selecting only the teachers who indicated that each activity has been used (frequency of use of each activity > Never used). Levene’s test was used to assess the equality of variances. Because of the larger number of comparisons in this step, we introduced a correction process to reduce the possibility of Type I errors. Bonferroni tests were applied to data showed in Table 4. A new adjusted significance level is used to evaluate the significance of all comparisons.

4 Findings

4.1 RQ1: What is the Frequency of Use of Moodle Activities in the Secondary Classrooms?

Table 2 shows the descriptive results of the use of Moodle activities in classrooms.

Table 2 Frequency of use of each Moodle component in classrooms (N = 132)
Full size table

The activities used by the highest number of teachers (above 50%) are varied, and encompass one activity of delivery (assignment, 78%), one activity of assessment (quiz, 69.7%), one activity of communication (forum, 69.7%), one activity of organization (lesson, 59.8%), and one activity of reusability (external tool, 53.0%). The activities used by fewer teachers (under 30%) are all related with peer-to-peer learning-tasks, and include one activity of delivery (workshop, 20.5%), one activity of creation (database, 28.8%), and one activity of collaboration (wiki, 29.5%). In addition, Moodle activities most frequently used (at least weekly for 25% of teachers) are assignment (50.7%), lesson (39.4%), external tool (29.6%), and forum (25%).

From the student’s point of view, these results mean that the main learning activities used on Moodle for the most students and with the most frequent development consist of uploading and submitting documents and receiving feedback from teacher, answering questions such as multiple choice or true–false, participating in virtual communities with peers, accessing structured content and answering questions about this content, and using external tools which could provide access to a new activity type, learning materials from a publisher, or a new software such as Skype, Google Hangout, Flickr, wikispaces, or Hot Potatoes.

4.2 RQ2: Do the Teachers Perceive that Some Relevant Factors About Teaching and Learning in Classrooms Can Be Affected by the Usage of Moodle?

Factor analysis showed an acceptable five-component structure (KMO = 0.884 and a significant Bartlett test, p = 0.000), and explaining 59.83% of the total variance. The resulting five factors derived from the survey’s items related to the perceived learning impact are: Promoting collaborative learning (12.99%), increasing quality in teaching and learning (32.73%), promoting inclusive education (4.47%), providing new educational scenarios (5.54%), and increasing engagement to learn (4.09%). The five components showed an acceptable reliability, with a Cronbach’s α of 0.706, 0.861, 0.694, 0.855, and 0.861, respectively (see Table 3).

Table 3 Structure matrix (factor loadings) and descriptive statistics for teaching and learning impact of Moodle use in classrooms (N = 132)
Full size table

Providing new educational scenarios includes statements mainly related to extend the variety of learning assignments, and to enable the teacher to design more innovative assignments as well as a greater diversity of learning experiences. Increasing engagement to learn refers to remarks such as increasing motivation, interest, attention, satisfaction and happiness in students’ learning. Increasing quality in teaching and learning highlights aspects which encompass both to make the teacher feel happier with the quality of his/her teaching, and to contribute to greater achievement of learning goals, learning outcomes and academic performance. Promoting collaborative learning refers to questions related to collaborative learning and joint development of learning products. Finally, promoting inclusive education includes accounts focused on student diversity and individual students’ learning needs.

Teachers scored higher on the factor providing new educational scenarios (M = 4.24) and, in fact, this is the only factor in which the mean is higher than the “agree” level. The other four factors are scored between “Neither agree nor disagree” and “agree”, in accordance with that decreasing order: Increasing quality in teaching and learning (M = 3.79), promoting collaborative learning (M = 3.68), increasing engagement to learn (M = 3.62), and promoting inclusive education (M = 3.60). From the learning point of view, these results mean that teachers perceived the last four aforementioned factors as being not greatly affected by the usage of Moodle.

4.3 RQ3: What are the Differences in the Perceived Learning Impact of Moodle Activities Among Users/Non-users of Each Activity?

Table 4 shows links between teachers’ perceptions of learning impact of Moodle and use of Moodle activities.

Table 4 Differences of teachers’ perceived learning impact of Moodle activities among users/non-users of each activity (N = 132)
Full size table

Altogether, Table 4 shows that users of only five activities perceived the overall index of learning impact of Moodle activities as significantly higher than non-users. The activities which show significant differences between users and non-users are database (t = 3.582, p < 0.0008), forum (t = 3.495, p < 0.004), glossary (t = 3.349, p < 0.004), quiz (t = 4.656, p < 0.0008), and survey (t = 3.371, p < 0.004).

Users of the database activity, as a creation tool, think that Moodle increases the engagement to learn (t = 2.947, p < 0.004) and promotes collaborative learning (t = 4.377, p < 0.0008). Users of the lesson activity, as an organization tool, think that Moodle provides new educational scenarios (t = 2.953, p < 0.004). Among delivery tools, which encompass the assignment and workshop activities, only users of the assignment activity perceive that Moodle increases significantly the quality in teaching and learning (U = 935.500, p < 0.004). In relation to the communication tools, which include chat and forum activities, only users of forum perceive Moodle’s learning impact as significantly higher on providing new educational scenarios (t = 3.063, p < 0.004), promoting collaborative learning (t = 3.211, p < 0.004), and promoting inclusive education (t = 3.325, p < 0.004). Among the collaboration tools, which include glossary and wiki, teachers’ users of both activities perceive Moodle’s learning impact as significantly higher on promoting collaborative learning (t = 2.959, p < 0.004; t = 4.122, p < 0.0008, respectively), and teachers’ users of glossary on promoting inclusive education (t = 4.636, p < 0.0008). In relation to the assessment tools, users of each activity (choice, quiz, and survey) perceive Moodle’s learning impact as significantly higher on promoting collaborative learning (t = 4.122, p < 0.0008; t = 4.314, p < 0.0008; t = 3.906, p < 0.0008, respectively). In addition, teachers’ users of quiz perceive Moodle’s learning impact as significantly higher on increasing quality in teaching and learning (t = 4.398, p < 0.0008), and promoting inclusive education (t = 4.972, p < 0.0008), and survey’s users also on promoting inclusive education (t = 3.119, p < 0.004). Finally, users of the external tool activity, as a reusability tool, do not significantly perceive any learning impacts.

5 Discussion and Conclusion

An overview of findings previously mentioned, which focus on teachers’ perceptions of the use of Moodle activities and their learning impact in secondary education classrooms, allows us to draw three interrelated conclusions.

In connection with the first research question, three types of Moodle activities (assignment, forum, and lesson), are simultaneously the most used and the most frequently used. Blin and Munro (2008) also found that the types of Moodle activities most used at a university level, over one whole course, were discussion forums followed by assignments. Why is there such a difference in the frequency of use among activities? It appears that these activities have three common features. First, the three activities enable the design of learning environments based on problem solving, learning inquiry, and complex, open-ended and ill-structured learning assignments (De Jong et al. 2012). Second, these activities address certain main roles of secondary school teachers in relation to instructional goals. Lesson and assignment are activities that allow teachers to provide subject guidance and students to develop their own skills and understanding of the subjects (Kelly et al. 2013). Third, these activities enable teachers to customize and adjust certain technological functions to the learning needs of students, including organization, delivery and communication tools.

The low number of users of communication and collaboration tools (excluding forum), as well as the low frequency of use of these activities, brings into question whether teaching practice in secondary education using Moodle is really in line with the social constructionism theory of learning which underpins the technological design of Moodle. These seemingly contradictory findings may be due to teachers’ beliefs that secondary students will most likely have a negative perception of collaboration using Moodle (Psycharis et al. 2013) in face-to-face scenarios. This belief makes it necessary to rethink the role that Moodle should realistically have in face-to-face educational settings in secondary education, particularly since it was designed to develop communication in online scenarios.

In addition, teachers perceived that the use of Moodle impacts only in one aspect of their educational practice in classroom: providing new educational scenarios. Teachers perceived that the only educational benefit of the Moodle’s use is to extend the limits of the physical classroom. At higher education, evidence about this issue is contradictory. Whereas activities for efficient communication among teachers and students is more highly valued by university teachers than interactive activities for innovating existing practices (Lonn and Teasley 2009), university students agree that the use of Moodle positively influences their perceptions of the quality of learning materials, the level of student performance and learning outcomes (Damnjanovic et al. 2015).

Finally, in relation to the third research question, only teachers as users of a narrow range of activities perceived the learning impact of Moodle as significantly higher: database (creation and sharing information tool), forum (communication tool), glossary (collaboration tool), and quiz and survey (assessment tools). In addition, users of a high number of activities (seven in total, the preceding five and also wiki and choice) perceived that Moodle significantly impacts promoting collaborative learning, which is the learning impact significantly mentioned by users of more activities. Both empirical data supports the assertion that secondary teachers who actually use these Moodle activities tend to use Moodle for instructional purposes in line with social constructional theory (Cole and Foster 2007). In addition, the activities that demonstrate the highest perceived learning impact are not the same as the activities most used in the classroom (such as assignment, lesson and external tool). It seems clear that the whole types of learning impact used in our study do not include all the reasons that are prompting teachers to promote the use of Moodle in classrooms. Whereas assignment, lesson and external tool were used by more than 50% of teachers, only the assignment was perceived as influencing learning in the sense of achieving learning outcomes. The other two do not seem to provide any learning impact for teachers as users.

In sum, our study contributes to clarify which is the real teaching and learning impact of the use of different types of technology affordances in the educational practice in classroom, in our particular case focused in the use of Moodle in secondary education. In this same research field, other recent contributions analysed the educational impact of the use of mobile learning (Gómez and Badia 2016), of augmented reality (AR) learning environments (Wu et al. 2017), and serious games (Iten and Petko 2016).

Our study has two main limitations. Firstly, our contribution should be viewed as an exploratory and preliminary study about secondary teachers’ perceptions of Moodle activities and their learning impact and uses in secondary classrooms. Further research would be required to extend data relating to students’ perspectives and, more importantly, the use of Moodle in real scenarios. Secondly, any future research developed about the use of Moodle activities in secondary education should take into account that teachers are able to use not only singular Moodle activities but also sets of combined activities in order to design complex learning assignments.

Our findings raise some questions that could be interesting from an educational research perspective. For instance, focusing on Moodle activities and Learning outcomes, one future research question could be: Why have teachers considered that only certain Moodle activities have a positive impact on learning outcomes? These questions may also be linked to practical concerns related to educational practice. Also related to learning outcomes, a practical challenge might be: what can technological designers and teachers do to improve the level of learning outcomes achieved through the use of Moodle activities? Therefore, future research on Moodle activities and learning outcomes should emphatically focus on answering those questions.

5.1 Note About the Possibility of Providing the Raw Data Without Identification of this Research

Data collected has been stored and managed observing the law on data protection and the right to confidentiality. Access to the database will be provided by the first author on the request of the interested part. Solicitations should contain information about the aim of the research and the type of analysis that researchers want to do. Applicants will be given a well-reasoned reply.