Introduction

A constant concern among teaching institutions and staff is to improve their students’ learning outcomes. This concern can require a change in their preferred teaching methodologies. Such changes could relate to the use of new information and communication technologies (ICT), often as a complement to conventional classes. ICTs complement traditional forms and means of education and can achieve a more effective and efficient means of learning (Lei 2010). One way of introducing this technology into teaching is through the concept of “blended learning”, which is defined as the integration of face-to-face teaching with online activities (De George-Walker and Keeffe 2010; Macdonald 2008) and which, as a teaching instrument, allows students to learn at their own pace (Osguthorpe and Graham 2003). The technology is widely used in university courses with large numbers of student, and especially in those courses typically taken during the first year of undergraduate study, which is an important period in determining students’ commitment to learning (Huon et al. 2007).

The use of digital technologies and online materials has generated diverse lines of research (Kuh and Hu 2001; Robinson and Hullinger 2008). Many studies have focused on the effectiveness of the technology as it has been employed as a complement to face-to-face instruction in the classroom setting, and on the determinants and predictors of the success (or failure) of this approach (Tamim et al. 2011). Very few such studies, however, have focused on higher education in particular (Carle et al. 2009; Lim and Morris 2009). A broad-based analysis has been made of ICT use—how much or how frequently the technologies are used (Chen et al. 2010; Kuh and Hu 2001)—but little consideration has been given to how and why it is used (Lei and Zhao 2007). Nevertheless, these elements are crucial to the final results achieved. Therefore, it is of great interest to the entire university community—including not just teaching staff, but also students and the institutions themselves—as well as to other institutions funding investments in technology to be aware of the impact of web-based learning technologies on learning outcomes when used as a complement to face-to-face learning (Chen et al. 2010).

In order to examine this question in depth, the present study considers the relationship between the academic results achieved by university students and the amount and quality of the use of technologies intended to act as a complement to face-to-face teaching from an empirical standpoint. This study was carried out using a sample of first-year students enrolled in courses related to business studies. The use of ICT in higher education requires first an evaluation of the contribution such tools can make to students’ learning outcomes, especially when they are used to complement the face-to-face learning experience (Ginns and Ellis 2009). In our study, participation in online activities was voluntary. Therefore, our first task was to analyze the extent to which the use of technology may have influenced the students’ academic results. Secondly, we sought to determine the influence of the different types of uses the students made of the online learning activities on their learning outcomes. Finally, we analyzed the possible impact on these results of other variables that are normally considered in the literature, such as the students’ academic background, class attendance rate and subject preferences, together with the impact of students’ interests in and commitment to their studies, in a more general sense. A combination of perception and performance data increased the validity of the measures (McNaught et al. 2012).

The remaining sections of this paper are organized as follows: in the next section, we review previous studies conducted on this topic and formulate certain hypotheses regarding the present study. In “Methodology” section, we describe the data employed and the methodology followed. We then present and discuss our results and draw some conclusions.

Literature review and formulation of hypotheses

Blended learning has undergone rapid development in recent years. This form of study involves the complementing of face-to-face instruction with e-learning modules, with the understanding that technology offers numerous opportunities to influence students’ learning outcomes (Resta and Laferrière 2007). According to prior studies, supplementing traditional classes with online activities has positive effects on student performance. Lim and Morris (2009) studied a sample of 60 undergraduate students enrolled in a learner- and program-evaluation course at a US university and observed a direct influence of instructional and learner variables on learning outcomes from a blended instruction course. O’Toole and Absalom (2003) obtained similar results when considering a sample of 72 graduating education studies students.

The incorporation of technology into teaching requires a financial investment, which is justified by the fact that ICT favors and affords more efficient and effective learning outcomes by and for students and can improve performance, including academic results (Lei 2010). This idea remains to be tested, however, as empirical studies of this question do not usually refer to the specific employment of ICT in a given academic subject and their use is normally measured in terms of perceptions (Cooner 2010; Lopez-Perez et al. 2011). Establishing a link between the use of technology and academic achievement—for instance, by considering students’ final marks—is fundamental to the provisioning of greater investment in education technology (Zhao and Conway 2001).

Technology can be applied for different uses, however, and not always for an academic purpose. For example, students might use ICT to prepare class materials or for personal use (Ward and Parr 2010). In the present study, we refer to ICT in the sense of technology that has been specifically developed to reinforce academic content, as a support to face-to-face instruction, and we examine how such technology might affect student performance.

In view of these considerations, this study focuses on the performance of online activities by university students. These activities constitute a useful tool for the teacher, as they enhance students’ understanding of the facts and concepts presented in class (Brothen and Wambach 2004), as well as for the students themselves, who can thereby test their comprehension of the class content.

An essential consideration is that of whether blended learning impacts on students’ final grades. On the one hand, we must distinguish between students who make use of online activities and those who do not. On the other, we must examine whether, among the former group, the characteristics of use are related to the final marks they were able to achieve.

Some students might perceive these tasks as additional or external to the principal activities that are evaluated and therefore choose not to participate (Orton-Johnson 2009), or to abandon them after a short period, or to use them in an inadequate way (Forsyth and Archer 1997; Johnson and Kiviniemi 2009). Appropriate use of online activities has been shown to significantly improve results (Grimstad and Grabe 2004; Huon et al. 2007). Furthermore, students’ increased use of online tools and activities is considered to be one of the best predictors of learning and personal development (Carini et al. 2006). Accordingly, we hypothesize that different degrees of the use of ICT may produce different learning outcomes. On this basis, we hypothesize as follows:

Hypothesis 1

Differences in the number of online activities performed by students are related to significant differences in the academic results they achieve.

After testing the impact of the level of use of online activities on students’ results, the next step will be to determine whether the characteristics of this use also influenced the results. Many publications have focused on the time (how much) and the frequency (how often) of the use of technology (e.g., Chen et al. 2010; Du et al. 2004; Kuh and Hu 2001). In this context, it is necessary to examine in greater detail the way in which the technology is employed (Goldenberg 2000). In this sense, we speak of “quality of use” (Lei 2010; Lei and Zhao 2007), with particular regard to the subject-specific use of technology (Lei 2010). Online activities can be addressed in different ways. If they are used as a mode of study, there might be no limit to the number of attempts allowed or the time able to be dedicated to the assigned tasks (Johnson and Kiviniemi 2009). In addition, feedback may be provided that indicates that the response or result obtained is or is not correct and, in the latter case, explaining why (Johnson and Kiviniemi 2009; Piccoli et al. 2001; Sitzmann et al. 2006). Thus, if the activities are intended to allow students to test their knowledge of the material and to become familiar with it—and not merely to help students in memorizing the results of the proposed tasks—a positive effect on results will be obtained (Brothen and Wambach 2001). The presentation of comments following students’ responses to questions has been shown to have a positive impact on their results (Brothen and Wambach 2004; Ryan 2006).

Although it has been shown that there is a positive relationship between the duration of use of technology, the frequency of use of technology and the learning outcomes achieved, in the context of blended learning—where online activities are employed as a work tool—and taking into consideration that various attempts may be made to resolve tasks—for instance, taking as much time as is needed or accessing appropriate reference materials as needed—it is the degree of understanding of the course content that is expected to have the greatest impact on learning outcomes. Moreover, Brothen and Wambach (2001) have shown in their analysis of a 29-student sample that no positive effect on results is obtained when students make use of online tasks to memorize the results of the tasks rather than to test their understanding of the material.

The possibility of contrary effects being obtained as the result of an inappropriate use of online activities led us to consider that the time employed carrying out online learning activities or the frequency of use of online learning tools might not be related to the results achieved. On the basis of these arguments, the following two hypotheses are proposed:

Hypothesis 2

The mean time employed by students in performing online activities does not influence the results achieved.

Hypothesis 3

The mean frequency of the use of online technology by students does not influence the results achieved.

Prior studies have demonstrated the positive effects of the quality of use of technology on students’ learning outcomes. Johnson and Kiviniemi (2009), for instance, examined the effectiveness of weekly reading quizzes as a means of improving exam performance for a sample of 159 undergraduate students in an introductory social psychology course at a large US state university. Forsyth and Archer (1997) reported similar findings when considering a sample of 207 students enrolled in an ‘Introduction to Psychology’ course. The students’ use of the Internet in order to gain a better understanding of the subject content and achieve the successful resolution of activities were both expected to have positive effects on the students’ learning outcomes. Accordingly, the following hypothesis is proposed:

Hypothesis 4

The degree of success in the online activities the students performed is positively related to the results they achieve.

According to previous research, other variables can influence students’ learning outcomes and therefore should be taken into account. Students’ prior academic results, for instance, could be a sound predictor of future academic performance (McKenzie and Schweitzer 2001; Rohde and Kavanagh 1996), although this has not always been found to be a significant variable (Huon et al. 2007).

A low rate of class attendance also might have a negative effect on performance (Paisey and Paisey 2004; Sugahara and Boland 2006). In addition, the relationship between blended learning activities and class attendance could indicate a greater degree of student commitment and perseverance and, moreover, this might affect the final marks the students achieve (Lopez-Perez et al. 2011).

Finally, students’ learning outcomes might be related, among other variables, to students’ perceptions (Schommer 1990). Specifically, Sugahara and Boland (2006) found that students preferences regarding the subject content—here, accounting—are related to their final marks, in that this interest might affect their commitment to the subject and thus influence the marks they ultimately obtain.

Methodology

Scope of the study and sample characteristics

The blended learning experience was put into practice in the ‘Introduction to Accounting’ course at the University of Granada, which is taken by first-year students in the following four undergraduate-degree courses of study: Business Administration, Economics, Business Science, and the double degree in Business Administration and Law. The class was taught to 17 groups, with a total of 1,128 students completing the course during the 2009–2010 academic year. A student was considered to have completed the course when he/she took the corresponding final exam. For the purposes of this study, the students were divided into two groups—one group that would complete online subject-related activities and a second group of students who would not. This second group is here considered the control. The teaching methodology involved a combination of face-to-face classes together with 30 activities designed to reinforce the concepts and content presented in the classroom and based on the Moodle platform. The students voluntarily completed these online activities using the web site created for the course. The activities were intended to help the students achieve a greater understanding of the content and concepts presented in the classroom setting. The website contained two groups of activities: first, individual activities for the student to help them better understand key concepts (e.g., crosswords, related words, fill-in-the-blank, multiple-choice quizzes) and to reinforce procedures (the accounting cycle); and secondly, participative and cooperative activities (e.g., wikis aimed at strengthening the students’ grasp of the key concepts and forums meant to underpin certain aspects of accounting procedures). The forums and discussions were coordinated by the teachers. Through autonomous learning, the students became more involved in the learning process and it was hoped that this would have a positive effect on their final marks.

The online activities were designed to be used as study tools and, upon completion of each, the student received an evaluation of his/her performance and thus was made aware of their progress; after each unsuccessful response, the system provided feedback explaining why the given response was incorrect (Johnson and Kiviniemi 2009; Piccoli et al. 2001; Sitzmann et al. 2006).

All teachers had the same syllabus and used similar teaching strategies and materials (e.g., manuals, slides and exercises). Several meetings were held to coordinate their teaching, to review the materials used or to add new ones (online or class activities) and to overcome potential difficulties. The teachers had been teaching this course for many years and had considerable experience in this field. Finally, to avoid any potential bias, the students’ exams were randomly distributed among the teachers for marking, irrespective of the particular group of students each teacher had taught.

For each group, and during the class, the teacher described the different activities on the website as they corresponded to the content given in class. The website was updated in accordance with the increasing body of knowledge the students had acquired. The teachers could also use these activities as a backup to the materials given in class and to observe the results obtained by the students in the different tasks. Thus, the teachers received up-to-date information on the class concepts and content that had or had not been understood and had the opportunity to reinforce these aspects in the classroom when necessary. Online activities complemented the traditional content of this course but were not sufficient on their own for the students to pass the final exam, and therefore cannot be considered substitutes for traditional classes and their content. The students still had to study using standard methods (e.g., slides, textbook and exercises) to pass the final exam. The online activities provided included games, quizzes, etc., about specific aspects of the subject, frequently those that were the most difficult to understand. These activities are useful for autonomous learning. Online tasks were voluntary; 598 (53.1 %) students completed them. Table 1 shows the percentage of students who completed the diverse activities, grouped into seven blocks, so that it will later be feasible to conduct an analysis of variance among the groups:

Table 1 Students grouped by the number of different activities completed
Full size table

Variables selected and methodology used

Learning results are multidimensional in nature. Objectively, they can be measured via academic achievements, by considering either improvements in learning habits or the skills acquired (Lei 2010). Previous studies have considered the academic achievements obtained by students as a good predictor of learning results (Drennan and Rohde 2002; Freeman and Capper 1999). In this study, we measured students’ academic achievements in terms of the final marks they obtained in the course (FINAL MARK). The primary aim of the teaching process is for students to learn the subject matter. An important aspect of this is their understanding of the course content; an objective way to measure this understanding is to use the mark the student obtained on the final exam. Accordingly, the latter variable was selected.

In evaluating the use and the quality of use of online activities, the following variables were taken into account:

  1. (a)

    Number of tasks completed (NTC). On the course website, students were offered 30 activities related to the material presented in class. In accordance with the established groups, this variable was assigned a value ranging from 0 to 30 depending on the number of activities completed (Johnson and Kiviniemi 2009). In addition, we created a dummy variable (ACTIVITIES), which took the value 1 if the students had completed the activities and 0 otherwise.

  2. (b)

    Duration of use (TIME). The average time spent by each student in completing the tasks (Chen et al. 2010; Kuh and Hu 2001).

  3. (c)

    Frequency of use (FREQUENCY). The mean number of times each student visited the course website to complete an activity (Du et al. 2004). This variable reflects the number of attempts, on average, the student made in trying to complete the activities.

  4. (d)

    Success in online activities. The mean score obtained for the activities completed (POINTS). The website provides information on the score each student obtained for each activity. This variable reflects the students’ average scores for the activities.

In addition, certain control variables were introduced—namely, the students’ prior educational achievements (BACKGROUND), rate of class attendance (CLASSES) and interest in the subject matter (PREFERENCE).

To measure the variable BACKGROUND, we used the average mark obtained by students on the university entrance exam, taking into account that the subject in question, accounting, was taught in the first year of study.

The class attendance variable (CLASSES) was included to determine whether this was a relevant factor in the final marks obtained. This variable was measured using the five-level scale employed by the University of Granada (<20 %, 21–40 %, 41–60 %, 61–80 % and >80 %).

Finally, we introduced the student interest variable (PREFERENCE), which was measured via a questionnaire on students’ perceptions of accounting. The answers were measured on a 5-point Likert scale, where 1 indicated “lowest degree of interest” and 5 indicated “highest degree of interest”, according to Sugahara and Boland (2006).

In order to test Hypothesis 1, we employed an analysis of variance to establish the differences between the groups and to determine whether there were significant differences in the results achieved by students who had not used the online activities versus those who had. We then applied a hierarchical regression model so that we could examine the effect of having performed the online activities on students’ final marks (Model 1):

$$ {\text{FINAL MARK }} = {\text{ b}}_{1} {\text{BACKGROUND}} + {\text{ b}}_{2} {\text{CLASSES}} + {\text{ b}}_{3} {\text{PREFERENCE }} + {\text{b}}_{3} {\text{ACTIVITIES }} + {\text{e }}\left( {{\text{Model }}1} \right) $$

To test the other hypotheses proposed in this study, the following hierarchical regression model was established (Model 2):

$$ {\text{FINAL MARK }} = {\text{ b}}_{1} {\text{TIME}} + {\text{ b}}_{2} {\text{FREQUENCY}} + {\text{b}}_{3} {\text{POINTS}} + {\text{ b}}_{4} {\text{BACKGROUND}} + {\text{b}}_{5} {\text{CLASSES}} + {\text{ b}}_{6} {\text{PREFERENCE }} + {\text{e }}\left( {{\text{Model }}2} \right) $$

We have used hierarchical regression because the control variables are temporal antecedents to activities.

Results and discussion

The results of the descriptive statistical analysis of the variables examined in this study are presented in Table 2. It can be seen that students who had not performed the online activities (control group) obtained an average score of 5.66 out of 10, while those who did obtained an average score of 6.99. On average, the latter group completed 17 activities, spent about 18 min on each activity, performed each activity 1.15 times and achieved an activity score of 8.27 out of a possible 10 points.

Table 2 Descriptive statistics of the variables used
Full size table

With respect to the control variables, the average university entrance mark obtained was close to 7/10 for both groups. Moreover, both groups attended around 80 % of their classes and showed broadly equivalent levels of interest in the subject (3.18 and 3.60 out of 5, respectively).

To test Hypothesis 1, we examined the data presented in Table 1 to determine whether there were differences between the final marks obtained by students who had not carried out any of the online activities and those who had completed a given number of said activities. We performed an analysis of variance, which showed there to be statistically significant differences between the different blocks of students (F = 23.848, DF = 6, p = .000). We can therefore conclude that there were indeed significant differences between the results achieved by students who had not performed any of the online activities and those who had. Therefore, Hypothesis 1 is accepted. It can be seen that the non-use of online activities or the performance of a different number of said activities gives rise to significantly different learning outcomes.

In addition, we proposed a hierarchical regression analysis. Table 3 shows the correlations between the variables included in Model 1 and illustrates the effect of the performance of online activities on students’ final marks.

Table 3 Correlations between variables (Model 1)
Full size table

The results show there are significant relationships among the variables used in this model.

The hierarchical regression results in Table 4 show that students’ final marks depend on certain variables, such as whether the activities were performed (ACTIVITIES), and on certain control variables—namely, the students’ marks on the university entrance exam (BACKGROUND), their class attendance rate (CLASSES) and their interest in accounting as a subject (PREFERENCE).

Table 4 Hierarchical regression analysis (Model 1)
Full size table

Previous studies, too, have associated the variables of background, class attendance and students’ interest in the subject with the learning outcomes achieved (Lopez-Perez et al. 2011; McKenzie and Schweitzer 2001; Sugahara and Boland, 2006). We found these variables to be significant in our model. Furthermore, the inclusion of subjective variables, such as students’ interest in the subject, increased the validity of the measures (McNaught et al. 2012).

The variable ACTIVITIES presents a positive and statistically significant relationship with the final marks obtained. The students who completed the online tasks obtained a higher final mark than did those who had not performed said activities. Both study groups had similar backgrounds, rates of class attendance and subject preferences, but those students who completed the online activities achieved better learning outcomes. These results can be accounted for as follows: these students dedicated more time to practice, they performed different activities that led them to understand the concepts in depth and they learned the subject in a progressive manner. They worked on their own and did exercises designed to develop other skills, such as reasoning and decision making, and so their learning process was more comprehensive. Their voluntary use of the website (Grimstad and Grabe 2004; Huon et al. 2007) and the greater commitment they showed to learning the subject—as displayed in their performance of additional activities proposed on the website (Carini et al. 2006)—affected their results. Therefore, by examining the use of online activities, we can identify differences in learning outcomes. The other students attended the class lectures but used only the printed reference materials, which are less flexible and more traditional, although nevertheless sufficient for some students to master the subject.

The next step in our study was to determine which of the variables related to the use and the quality of use of online activities affected the students’ final marks.

Table 5 shows the correlations between the variables included in the hierarchical regression model used to test the other working hypotheses. This table shows that the students’ final marks were positively and significantly correlated with the scores they obtained in the online activities (this variable was used to test the quality of the use of the technology). The students’ final marks presented a positive association, which was also significant, with their background knowledge, their class attendance rates and their stated level of interest in the subject. Furthermore, there was found to be a negative and significant correlation between the final marks achieved and the time dedicated to each of the tasks.

Table 5 Correlations between the variables used (Model 2)
Full size table

In relation to the control variables, there was found to be a negative and significant relationship between the time employed by the students in activities and their background, which might mean that students with less prior knowledge will make increased use of this type of supportive resource. There was also found to be a positive relation between the score obtained in the online activities and the students’ prior knowledge, which suggests that students with a better grounding in the subject obtained better results in the online activities. The results revealed the presence of statistically significant relationships among the control variables used in this study. There was also found to be a positive and significant association between a higher level of prior knowledge and a higher rate of class attendance.

Table 6 shows the results of the hierarchical regression model proposed. This model shows the effects of the variables related to the use and the quality of use of ICTs and the control variables on the final marks. It can be seen that this model explains students’ learning outcomes (FINAL MARK).

Table 6 Hierarchical regression analysis (Model 2)
Full size table

The hierarchical regression results included in Model 2 and presented in Table 6 show that the time devoted to working with the online activities and the frequency of their use are not explanatory factors of the final results achieved, and thus Hypotheses 2 and 3 are accepted. The fact that these variables are correlated with others in the model means that, when considered jointly, only those most relevant to the model are significant. In this respect, it can be seen that the quality of the use of the technology, which is measured as the score obtained in the online activities, is a significant predictor of the final marks obtained after controlling for the temporal antecedents to online activities (e.g., BACKGROUND, CLASSES and PREFERENCE). The students’ scores on the online activities reflect their level of understanding of the content. It is possible that the use of such tools is able to enhance students’ in-depth knowledge because students perform the activities over an extended period of time. Accordingly, these findings may support Hypothesis 4. The use of the Internet in order to achieve a greater understanding of the subject and the successful completion of online tasks did have a positive effect on the final marks obtained (Johnson and Kiviniemi 2009). Therefore, the scores obtained both reflect an academic result and the fact that, as a result of having completed the activities, the students developed skills such as reasoning, problem solving and decision making that helped them obtain better learning outcomes. Although we accept this finding, we do recognize that there may be other explanations—for example, students could have improved their test-taking abilities or there may be a relationship between the questions asked in the online environment and the final exam. Web-based technology provides a backup and a complement to face-to-face learning and has a positive effect on the results achieved when the students are committed to the process (Chen et al. 2010). These online activities, which were presented as voluntary, had a positive effect on the final results, as has been reported in other studies (Grimstad and Grabe 2004). The results obtained in the present study support and justify prior investments made in using information technology to outperform university students’ involvement and achievements (Carle et al. 2009; Forsyth and Archer 1997). With regard to the control variables, greater prior knowledge, more regular class attendance and a greater interest in the subject all have a positive effect on the final results.

Conclusions

Our analysis shows that the use of online technology and different types of said use—for instance, as a backup and a complement to the content disseminated in a face-to-face classroom setting—may produce different learning outcomes among students as measured on their final exam results. Students’ use of online resources to learn by participation and their completion of online tasks enables them to acquire knowledge and develop skills such as reasoning, problem solving and decision making, all of which lead to them achieving better marks. Although there may be other explanations for these results, the quality of the use of technology is a significant predictor of the final grades. The present study shows that having performed these activities has a positive impact on the results students obtain, in the form of higher final marks.

In conclusion, ICTs facilitate positive results when they are used to understand the content and concepts of the subject, and not merely to facilitate memorization of content. Online technology allows students to learn gradually, autonomously and at their own pace and the results demonstrate that this has a positive effect on their progress. However, our results also show that the mere use or the frequency of use of online learning is not significant.

It is necessary that the online activities be appropriate in their design in order for satisfactory results to be achieved. A greater understanding of course content is obtained from activities that include an element of feedback, for instance by explaining why certain responses are incorrect. Such design provides students with autonomy in determining their work rate and enables the teacher to reinforce those areas in which the students continuously obtain poor results.

This study has shown that voluntary activities can provide additional course materials that are useful for students who want to expand their understanding of the subject matter or to attain understanding at their own pace, and that this has a positive effect on the students’ final results.

Our findings have lead us to conclude that greater involvement on the part of students and a greater degree of commitment to and interest in the subject are crucial factors in determining the students’ achievements. Students’ interest in the subject matter, their rate of class attendance and their voluntary performance of online tasks, all intended to enable students to gain a greater understanding of the course content and to study at their own rate, are positively related to the final results obtained. In addition, the degree of prior understanding is a significant variable. It is also important to note that the negative and significant relationship between the time employed by the students and their background knowledge of the subject could indicate that students with poor prior understanding make greater use of this type of teaching resource. Positive effects are gained from providing such materials to students who want to perform a greater number of tasks in order that they might better understand the subject content.

The most valuable finding of this study is that the performance of supplementary online activities can enhance the learning process and contribute to an improvement in students’ final marks. This type of learning might be an important instrument that allows students to complement their studies and could also decrease the dropout rate (a significant problem in higher education). These findings may encourage teachers to use the described approach in order to improve learning outcomes.

The present study does carry with it certain limitations. We did not take into account those students who did not complete the course, although some of them made use of the online resources. Analysis of this factor would improve our understanding of the results.