1 Introduction

Blended learning, a combination of face-to-face and computer-assisted pedagogy, is gaining in popularity as an alternative delivery mode for university courses (Lindorff and McKeown 2013). To date, the majority of research studies have been involved in augmenting a face-to-face class with online work, not replacing face-to-face class time, or totally redesigning a face-to-face class to include blended learning. This research assesses the value of a blended management information systems course in which one-third of the class meetings were replaced with online activities and resources. Unique to this study is that no past research has compared a traditional class with a blended format with the same content and same professor, holding the classroom exercises constant but online. Formative assessment was performed to obtain affective and perceived indicators of success. Results from summative assessments of learning outcomes were compared to results from previous classes using a critical realism lens therefore enriching the research by interviewing students. A model for blended learning as an improvement for the future was developed from the quantitative and qualitative data collected.

The research questions driving this study:

  1. 1.

    For a true blended learning class, with holding the content and the lecturer constant and varying only the class modality, what are the changed outcomes?

  2. 2.

    Considering this particular research situation, does this different learning environment (blended) contribute to superior learning as measured by assignment, exam, and final course grades?

  3. 3.

    Is there a significant difference in learning for international students who may not have their mother tongue language in the language of instruction?

The methodology of this research in management information systems can be applied to all types of management undergraduate courses. The results predict, although mostly are neutral or positive, that there are some negatives to teaching with a blended learning modality.

1.1 Research project goals

Modern technology has given faculty new ways to incorporate active learning and increase student engagement in their courses. The rationale behind this research can be summarized as follows:

  • To be able to compete in today’s fast-paced competitive business environment, students need to be self-learners and lifelong-learners. A blended learning approach puts more of the learning onus on the student and helps them prepare for future careers.

  • Past research in other class subjects has shown that students learn more and are more satisfied in some blended learning environments (Alrushiedat and Olfman 2013; Osgerby 2013). However, each study has limitations, and many are for classes that have simply been augmented with online work. This research addresses some of these limitations.

  • Past research relies on classes that have been totally redesigned, resulting in more learning time (Means, et al. 2013) whereas this project held constant all content varying only one-third of the modality.

  • Blended learning is scarce in business courses (Arbaugh et al. 2009).

  • On some campuses, including the site of this study, classrooms and computer classrooms are in high demand (Tynan et al. 2013). A blended learning class would help address this resource scarcity and perhaps allow for more students to enroll in courses, thereby potentially improving retention and graduation and utilizing classroom space more efficiently (Owston et al. 2013).

  • Research has shown that blended learning has the greatest impact on those students who are already computer-literate (Yudko et al. 2008) and that student’s “academic ability” is a critical success factor for blended learning (Owston 2013). A second-level management information systems class should prove a good test case for this type of class.

  • Students from different backgrounds with different levels of fluency in the language of instruction deserve an equal opportunity to obtain an education. Some research has shown that blended learning allows for students with divergent requirements to achieve their potential at their own pace using common and well-known online tools (Keith and Simmers 2013).

  • Past research studies call for “critical self-reflection” (Garrison and Vaughan 2013:25) in the implementation and success of blended learning.

  • Many previous studies looked at online programs and commuter programs in city universities (Owston et al. 2013; Castle and McGuire 2010). This research study is based on the traditional out-of-high school undergraduate who lives on campus.

The goal of this research project was to examine how well the students met the learning outcomes from a blended learning curriculum and to develop a model for blended learning. Although blended learning takes the form of many different deliveries, for this research project it’s defined as a course structure that replaces a significant portion of the traditional face-to-face classes with online activities. This encourages student learning autonomy. This assessment project serves as a model for future implementations of blended learning across the entire college’s course offerings. This paper reviews the literature on assessing blended learning outcomes in higher education and describes new research in progress. Section 2 discusses major learning theories and the published literature on blended learning. Building on that literature, Section 3 describes the research organization and results. A discussion of those results follows in Section 4. The study limitation, future work, and major conclusions of the paper are then drawn.

2 Learning theories and blended learning

The basis for this research, interview protocol, and interpretation of results is the constructivist learning theory. Blended learning in this research is defined as replacing a portion of the face to face class time with online classes and assignments.

2.1 Learning theories

There exists a rich set of learning theories, but the one that can be best associated with blended learning is psychological constructivism. This constructivist view takes the student as an active learner, built on a foundation of prior knowledge and experience. The learner shapes their own knowledge and demonstrates that understanding by solving real-world problems. Personal experience influences knowledge construction. Since students are creating their own knowledge and their own understanding, they are better problem solvers (Al-Huneidi and Schreurs 2013; Duffy and Jonassen 1992; Cunningham 1992). Instead of the educator feeding the students information through the traditional lecture style, students are active and dynamic learners. Problem solving is at the heart of learning to construct one’s own knowledge, with professors supporting that knowledge construction.

Constructivist history began with Jean Piaget’s ideas that “children build and develop their own intellectual structures through interaction with the environment, and that instruction can only follow, and not promote, that personal intellectual development” (De Corte 2003:110). Piaget felt that learning is active and dynamic, absorbing knowledge, creating and testing theories. John Dewey also took a similar approach in developing his spectator theory. One can attempt to learn a game by being a spectator, but to truly learn, one must also play the game. Rote learning is not the way to teach; students must experience in order to learn. (Phillips 1995).

In today’s competitive, complex, and fast-paced work environment, students who are prepared well will thrive. By students constructing their own knowledge, they are better prepared for this rapidly-changing business workplace (Al-Huneidi and Schreurs 2013). Specifically, in the management information systems curriculum, students need to solve business problems with computer systems. By promoting a constructivist theory, educators are enabling students to be better problem-solvers in complex computer models. The complexity of computer programming, for example, is best learned by the student creating their own knowledge which leads to “higher order thinking skills” (Hadjerrouit 2008), not simply passively taking notes.

2.2 Blended learning

Multiple definitions of blended learning exist but the predominant idea is that blended learning is a combination of face-to-face and online computer-assisted pedagogy (Hamilton and Tee 2013; Bliuc et al. 2007). Hybrid learning is synonymous with blended learning and the labels are used interchangeably within the literature (Martyn 2003). There isn’t one set proportion of face-to-face lectures to online lectures but blended learning sits on a continuum, with varying degrees of information and communication technology (ICT) (Jones et al. 2009; Gomez and Duart 2012). This continuum permits professors to structure the amount of the ICT applied within a course. In addition, the continuum also allows for universities not using blended learning to begin to incrementally implement the strategy. The continuum is shown below in Fig. 1.

Fig. 1
figure 1

Blended learning continuum (Graham et al. 2013:5 . Copyright 2013 by The Internet and Higher Education. Reproduced with permission via Copyright Clearance Center)

Full size image

In this research, the authors define blended learning as that of Garrison and Vaughan (2013) a modality that actually replaces some of the traditional face-to-face class time with asynchronous online course activities. It’s the “delegation of control to the student over the use of the technology” (Hartono et al. 2015:3) that allows students to do the work when they want, where they want, and encourages them to be self-learners which hopefully leads into life-long learners (Ibid). Institutions of higher education are moving towards blending some of their courses for a variety of reasons, mostly in hopes of enhancing learning (Hamilton and Tee 2010). At these institutions, faculty strive to facilitate true and lasting learning, not simply rote learning. Biggs describes two types of learning: surface and deep (Biggs 2003). Surface learning is indicated by memorization and regurgitation of the facts without any in-depth understanding. From the student’s perspective, surface learning can at times be caused by anxiety and time pressure stress. Deep learning, on the other hand, attempts to get at true understanding of the material and connect related material from other courses together. Most importantly for the concept of blended learning, “deep and surface approaches to learning describe the way students relate to a teaching/learning environment; they are not fixed characteristics of students, their academic personalities so to speak” (Biggs 2003:17). Blended learning opens up new possibilities in the teaching and learning environment, perhaps eliminating some of the anxiety and time pressure stress of a prescribed set class time. Research has supported this and identified changes in the mode of pedagogy that can shift the learning experience and outcomes (Hamilton and Tee 2010). Similar to other so-called “high impact practices” that connect the classroom with the real-world (e.g., internships, capstones, study abroad) (Kuh 2008), blended learning can also extend beyond the classroom to embrace different learning approaches which provoke critical thinking (Bonk and Graham 2005; El-Deghaidy and Nouby 2008). Blended learning can require significantly increased time-on-task on the part of students (Bonk and Graham 2005), opening the door for an improved learning experience.

Although the broad history of technology-enhanced coursework has demonstrated that student learning is usually very comparable to what occurs in traditional coursework, recent studies focusing specifically on blended learning report positive results. However, in many of these studies, face-to-face classes were simply augmented with online material and classes were completely redesigned to include blending.

“Studies using blended learning tended also to involve more learning time, additional instructional resources, and course elements that encourage interactions among learners. This confounding leaves open the possibility that one or all of these other practice variables, rather than the blending of online and offline media per se, accounts for the particularly positive outcomes for blended learning in the studies included in the meta-analysis.” (Means et al. 2013:36)

A brief summary of these more-recent studies in highly ranked journals is displayed in Table 1. Within most studies, all courses were at least equivalent to face-to-face courses and often had notable positive benefits linked to their use of blended learning.

Table 1 Recent literature on blended learning
Full size table

This review of recent literature contains surprisingly few results on grade outcomes with blended learning and few studies on truly blended classes. Most of the studies cited display positive results with using a blended learning modality, but only three of them actually replaced some of the face-to-face traditional classes with online course material (Owston et al. 2013; Demirer and Sahin 2009; Hamilton and Tee 2013). The rest of the studies measured students’ learning or feelings by adding complementary, supplementary, or optional online class work.

One theme in some studies is that strong students take full advantage, and even thrive, with the blended modality. A major study at a city university was able to survey students in 11 classes on the topics of enrollment, experience, engagement and learning. Their results show that high achievers were more satisfied and engaged with blended learning and low achievers preferred face-to-face (Owston et al. 2013; Farley et al. 2011). High achievers also appreciated blended learning to ease their commute and its expenses. The study by Ellis and Bliuc (2016) also demonstrated that deep-learners excelled in the online material, so the outcome of the class depended on their level of inquiry. Collaborative groups’ success with blended learning depended on having deep-learning members (Ellis et al. 2016). Likewise, when students are suitably prepared for the learning environment, blended learning can be more complex and more of a 2-way interaction rather than simply teaching to the student (Hamilton and Tee 2013). Challengingly, the majority of students may not be high-achievers and educators need to focus on how to satisfy all students (Biggs 2003).

As previously mentioned in Fig. 1, some universities deliver blended learning at different places on the continuum. By simply adding supplementary online material, studies have shown an increase in student grades and fewer students dropping the class (Means et al. 2013; Lopez-Perez et al. 2011, 2013; Du 2011) and improving attendance and satisfaction (Stockwell et al. 2015; Osgerby 2013). Demirer and Sahin randomly assigned education students to different classes, one using blended learning and one relying on traditional face-to-face instruction. Although they saw no statistical difference between the two classes in terms of the achievement tests, the scores on the multimedia projects from the blended learning group revealed a greater transfer of learning (Demirer and Sahin 2009). Management students in a blended learning class were surveyed and indicated that they liked the supplemental interactive tutorials and preferred them over the online lecture material. However, they concentrated on viewing the online material that was going to help them achieve good exam scores and not the material that might help them understand better (Lindorff and McKeown 2013). Educators want students to understand, not simply work towards good grades.

In a study of Chinese students learning the subject of quantitative marketing in English, the blended learning students reported greater satisfaction with the class partly because they were able to view the face-to-face lectures that had been recorded and provided in video format multiple times. The group of students in a traditional class, those that did not have access to the recorded video lectures, did not perform as well as the blended group on the exams (Keith and Simmers 2013). This particular research result argues that there is significant potential for blended learning to be explored with those universities who have a large population of students for whom the language of instruction is not their first language.

Some of the more recent studies offer implications for blended learning course design. Scaffolding of important class activities and molding students’ perception of learning encouraged engagement, a necessary criteria in today’s university educational assessment (Manwaring et al. 2017). Requiring collaborative group work and ensuring that the groups contain a deep-learning student (Ellis et al. 2016) are suggested as aiding in blended learning curriculum. Designing for active learning (Bower et al. 2015) and requiring online discussions (Alrushiedat and Olfman 2013) promote further engagement with blended learning students. To ensure self-learning, encouraging time-management strategies (Broadbent 2017), and teaching self-control from a young age (Zhu et al. 2016) contribute to positive outcomes within blended learning classes.

3 Research hypotheses, method, and results

The class, Business Information Systems, is the second level Management Information Systems (MIS) course taken by undergraduate business students in a business college of a large research university in the United States. Topics in the course include database design and implementation, advanced spreadsheets, ERP systems, business processes with flowcharts, and data analytics.

While the course is constantly being revised to stay technologically up-to-date, its delivery system has remained the same since its inception over 25 years ago. Historically, students attended face-to-face classes 2 or 3 times a week to listen to lectures, participate in discussions, or work on systems in the computer lab. For this study, the class met face-to-face on Mondays and Wednesdays with asynchronous online activities on Thursday and Fridays. The students were given flexibility as to when they completed their online work – any time between 8 am Thursdays until 3 pm Fridays. Assessing the impact of this shift in the pedagogical technological approach is the core of this research. It stands out among past research studies because both the content and professor are held constant in order to see the true change with adopting a blended learning approach.

Based on prior research, it is expected that there will be no difference in final grades between the non-blended class and the blended class.

  • H0: The final grade outcomes for both the non-blended class and the blended class will be the statistically equivalent.

  • H1: The final grade outcomes for students will be different for the blended class.

This research study was conducted in conjunction with the university’s teaching and assessment and opens the door for future research on international students’ outcomes. Using past known data such as international language skill level and the outcomes of the business information systems class, predictors of success may be determined.

3.1 Research method

The assessment in this study is motivated by the research philosophy of critical realism, a philosophy that attempts to understand the mechanisms underpinning causal relationships (Danermark et al. 2002). Under this philosophy, the researcher cannot use empirical results alone to determine theory. In a critical realism research study, mechanisms are hypothesized to describe the quantitative and qualitative data (Danermark et al. 2002; Bygstad and Munkvold 2011). Powers for (causal powers) and against (liabilities) the mechanism work to illustrate how the data acts in the particular situation (Sayer 1992). So the goal of critical realism is to come up with mechanisms that cause the outcomes of the data being researched (Bystad and Munkvold 2011) within the context of those involved.

Following the critical realism approach, this research study used a mixed methodology. Data were collected quantitatively from the blended classes (147 students) taught in the fall semester and, as a comparison, classes taught in the fall and spring semesters of the prior 2 years (413 students). The comparison group was taught by the same instructor with the same content in a traditional face-to-face environment, making this research study unique. Although the mode of delivery was changed from some course content, care was taken to ensure that the content of the class did not change. Students’ final grades in the two types of classes, traditional and blended learning, were analyzed by multiple regression models. In order to enrich the study and move beyond the quantitative analysis, concurrent with the course and its graded components were semi-structured interviews with several blended learning students. These interviews were conducted with the same students three times throughout the semester to collect affective and perceptual information for both formative and summative use. All of these data were analyzed in a stepwise fashion of conceptual analysis and causal analysis following a critical realist approach. Only the blended learning students were interviewed since the face-to-face students had already finished the class.

The combination of both quantitative and qualitative data collection and analysis allows for better understanding of the complex issues surrounding learning. If only one research method is used, then the researcher may only be looking at one aspect of the project, and missing out on other facets, showing limited results (Orlikowski and Baroudi 1991). The triangulation of methods, quantitative and qualitative, used in research makes it a stronger research project because the different methods can explain different aspects of the project and fill in any holes (Mingers 2001; Orlikowski and Baroudi 1991).

3.2 Data analysis - quantitative

The final grade scores of students in both the blended and traditional classes were compared with a number of other factors that might have an impact on the outcome as shown in Table 2. It was decided that these factors would be investigated since the data might provide insight as to which students are learning with or without blended learning and it might drive future research into this pedagogical modality. The data was obtained from the university central system. The final course grades were correlated with gender, whether the student was international, whether the student’s major(s) was in the business college, the student’s class, when the student took the Intro to Computing class (an introduction to computing concepts, spreadsheets and databases which is a prerequisite to the MIS class studied in this research), whether the student is a member of an underrepresented minority (URM) population (defined as not White and not Asian-American), and the student’s grade in the Intro to Computing class.

Table 2 Variables and frequencies considered in statistical analysis
Full size table

Our first research question focused on the learning that occurred in the blended and non-blended courses. We use the final course grades as our means of comparison, a reasonable comparison given the relative stability of the course over time as it has been taught by the same professor. The average grade in the blended course sections was 3.12, slightly lower than the average grade of 3.24 in the non-blended course sections.

Our second research question focused on the potential for improved learning in blended sections for international students. To answer this question, we began with a bivariate analysis comparing final grades in the MIS courses for international and domestic students (Table 3).

Table 3 MIS course grades for domestic and international students
Full size table

Next, we constructed a multiple regression model predicting students’ grade in the MIS class. The independent variables used in the models, first introduced in Table 2, are listed in Table 4.

Table 4 Description of independent variables used in the multiple regression models
Full size table

Students’ course grades in the previous Intro to Computing course were compared to their grades in the MIS course. The two classes are a sequence for most students and are designed to build upon one another, so we would assume there is a positive performance relationship between the two classes. The correlation between the two grades is .501, a moderately strong positive correlation between the two grades. Any refined model examining the relations of blended and non-blended performance should include the Intro to Computing grade. The remaining variables are included in the models for practical reasons as they are critical concerns for our university administration, faculty, and student body.

The R2 for Model 1 was .27, or 27% of the variability in students’ grade in the MIS course is explained by the model. As shown in Table 5, only two independent variables were statistically significant in the model: Intro to Computing and International. The coefficient for Intro to Computing is positive indicating students who did better in Intro to Computing also performed better in MIS (p < .001). Based on the dummy variable test, international students did better than domestic students. The difference in grades was not large, .289 on a 4-point scale, but it was statistically significant (p = .005, two-tailed test). The Blended variable was negative, but not statistically significant. This supports the naïve bivariate analysis that found little evidence that blended students performed substantially differently than non-blended students, therefore failing to reject H0. The grade outcome for the non-blended and blended classes are similar. However, the presence of the International variable as a significant predictor indicated that further exploration was warranted so we constructed separate multiple regression models for the students in the blended and non-blended course sections.

Table 5 Multiple regression model coefficients
Full size table

Model 2, the model with only the students in the blended course sections, has more explanatory power than the other two models with an R2 of .43. As in the previous two models, the Intro to Computing grade is statistically significant (p < .001, two-tailed test) with a positive coefficient. Unlike the other two models, the International variable is not significant. However, the Gender variable is significant with a small positive coefficient of .219 (on a 4-point scale).

The R2 for Model 3 with only Non-Blended students was .24, or 24% of the variability in MIS is explained in the model. Only two variables were statistically significant in the model, Intro to Computing and International. The coefficient for Intro to Computing is positive indicating students who did better in Intro to Computing also performed better in MIS (p < .001). Based on the dummy variable test, international students did better than domestic students. The difference in grades was not large, .307 on a 4-point scale, but it was statistically significant (p = .007, two-tailed test).

Although not reported here, we performed due diligence in testing all three models for the standard assumptions of multiple regression. Common tests and indicators – variance inflation factors, plots and analyses of residuals, normality of continuous variables, etc. – indicated no concerns and satisfactory fulfilment of the method’s assumptions.

3.3 Data analysis - qualitative

In order to fully understand the effects of blended learning on constructivism, it is imperative to move beyond the empirical results and look to qualitative analysis to understand the mechanisms of causality and the powers and liabilities that determine the workings of that pedagogical mechanism. Unfortunately, only the blended students were available for interviewing, but this data is still relevant in that a model for learning can be constructed within the context of this class.

Qualitative data was analyzed by conducting a theme analysis of the interview transcripts. The steps to this research are (1) develop interview questions; (2) interview students three times during the semester (note: interviewing was performed by the center for teaching and assessment of learning, not the instructor); (3) conduct a frequency count of objects from the blended learning literature heard in interviews. Every time a student mentioned a particular theme, it was marked and those marks were tallied up; (4) identify new objects in interviews. If students repeatedly mentioned new ideas, they were also recorded and tallied up; and (5) develop learning mechanism from the blended learning literature objects and from the new objects articulated in the interviews. These steps are now explained in detail.

Firstly, students were interviewed in three successive sessions throughout the course: once at the beginning (11 students), once in the middle (9 students, two dropped out), and once near the end (the same 9 students). Students in the blended learning class volunteered for the interviews and were compensated with gift cards for local businesses. To reduce potential bias that could dissuade student participation and impact their instructors’ assessment of them, the interviews were conducted by two of the researchers who were not involved in teaching this course. Interview transcripts were not shared with the course instructor until the semester ended and final grades were submitted and the identities of the interview subjects remain confidential and unknown to the course instructor.

Table 6 outlines the questions asked in the qualitative data collection.

Table 6 Interview questions
Full size table

In this research, there was a control group (those classes without blended learning) which helps alleviate internal validity threats (Cook and Campbell 1979). To have external validity, the outcomes of the study should yield generalizations for other groups, usually requiring random samples (Cook and Campbell 1979). Since the outcomes of this study are applicable in higher education at the undergraduate level and the subjects being used for this study were undergraduate students in higher education, external validity should be acceptable. Students should be representative of other university students.

An additional type of validity is construct validity, which sets out to ensure that there are no other constructs in the experiment which are causing the effect; in other words confounding (Cook and Campbell 1979). Although both the professor and the course content remained the same, one confounding variable in this study is that the courses were taught in different semesters. Since this is, in some ways, like a field experiment, so the semesters cannot be held constant, these confounding variables cannot be eliminated.

The interview questions’ validity was tested in a survey context by administering additional questions and comparing responses, asking other researchers and scholars for feedback on the questions, and comparing these questions with questions on existing instruments. Based on these criteria, these research questions are reasonably valid for this study. All interviews were transcribed and selected statements were coded using the categorical objects and their properties predetermined in the literature review. These predetermined properties are:

  1. 1.

    Enhanced, deep learning

  2. 2.

    Anxiety eliminated; less stress

  3. 3.

    Critical thinking

  4. 4.

    More time on task

  5. 5.

    Greater transfer of learning

  6. 6.

    More complex than face-to-face

  7. 7.

    Better for non-native speakers

  8. 8.

    Concentrate only on exam material

  9. 9.

    Self-regulatory skills; learning autonomy

For all three sets of interviews (beginning of semester, mid-way through the semester, and at the end of the semester), a frequency count was constructed with the objects that were mentioned in at least 50% of the interviews becoming necessary objects. These objects are now considered necessary because they are essential to understanding blended learning (Sayer 1992), for example, self-regulatory skills and learning autonomy in this research setting. This research not only relies on what the literature is providing but also has added new objects, being identified in the interviews, some becoming contingent aspects, meaning that they may produce a different outcome depending on the situation. These new objects can force different outcomes of the mechanism based on the context or contingent causality (Smith 2006; Bystad and Munkvold 2011). Known as contingent aspects they influence a mechanism positively or negatively. In the data they were identified as being prevalent in at least 25% of the interviews.

From the frequency count shown in Table 7, along with new objects of (1) lack of connectivity between online and face-to-face classes; (2) easier to ask questions face-to-face; (3) less distractions with face-to-face; and (4) better to read than listen to lecture with terms, the mechanism for understanding blended learning has been produced:

Table 7 Frequency count of predetermined objects
Full size table

This mechanism for understanding blended learning begins with students feeling less anxious and less stressed about having a new class with a portion online. These interviewees see themselves challenged to regulate their own work and learn on their own, a key enabler of the transfer of learning. This idea of self-regulation continues into the middle of the semester as well. By the end of the semester, they are fully wrapped up in the complexity of a blended learning modality. The forces that influence this model positively or negatively were extracted from the new objects that students mentioned in their interviews. Working against the learning in the blended environment was firstly a lack of coherence between the online work and the face-to-face work. Students felt there was a dividing line that was disconnected between the two parts of the class. They overwhelmingly preferred the face-to-face classes because they could ask questions in those classes, as opposed to sending emails to the professor when online. They also claimed that there were too many distractions at home, so they were more focused in the face-to-face class which worked against the idea of a blended environment. On the positive side, some felt the material that was online, readings and quizzes, were better online than having to listen to the professor lecture on terms via PowerPoint.

The overall theme for the interviews conducted in the middle of the semester was time management. Almost every student said that it was difficult to catch up once they fell behind in course work. One emphasized self-discipline and claimed that students think that online work or class is “free time” although it’s not. One student did not like the online classes, was struggling, and said that he’d rather go to (a face-to-face) class. So the concept of complexity in the blended learning environment was beginning to surface.

The final round of interviews, conducted towards the end of the semester, produced some interesting remarks. A number of students felt that they learned more in class because there were fewer or no distractions and during the face-to-face classes they were able to ask questions easily. One interviewee called face-to-face classes their “class zone” meaning they didn’t have the distractions that online social media presents. The challenges of learning online were articulated. As in the beginning and mid-semester, a few students even continued their theme of autonomy. One student remarked that you can retain more because you have to grasp information on your own and it’s harder but more concrete. Another said that she can teach herself more thoroughly because someone in class will tell you what to do but actually learning it on your own is better.

Throughout the interviews, students also mentioned the importance of faculty availability during the online portion of the class. They felt that the online material needed relevance, and that the material needed to be organized well for blended learning to run smoothly. Interestingly, one student commented that although they preferred face-to-face classes, 1/3 of the classes online was acceptable, but one half of the face-to-face classes would be unacceptable.

4 Discussion

Unlike previous classes that met three times each week, this blended MIS class replaced one of those meetings with online content and activities. Based on the qualitative analysis using a multiple regression model, this change in the mode of delivery does not seem to have affected the students’ final grades adversely. The average final grade for students in the blended learning MIS classes was slightly lower than the final grades of students in non-blended classes, but not statistically significant. One significant, positive predictor of grades in the MIS classes was the final grade in the previous class, Intro to Computing. Contrary to our expectations based on past research (Keith and Simmers 2013), international students’ did not earn higher grades in the blended course once we controlled for other variables in our second multiple regression model. There were some factors that did display an impact during the blended learning classes. Male students improved their final grades in MIS with the blended learning classes.

A qualitative approach was also used to help make sense of the complex learning that students undertake with a blended learning class. By developing the interview questions from the literature review, interviewing the students, analyzing their responses and adding any further ideas from the interviews, a mechanism for learning was developed. This blended learning mechanism shows that in the beginning and middle of the semester, students reported in interviews a greater awareness of the planning and self-discipline necessary to succeed in this course where much of the activity takes place online without direct, physical oversight by faculty and peers. So with the blended and flexible schedule, students may be learning more, spending more time-on-task, and increasing their self-regulation skills. On the other hand, the challenge of the complexity of learning by ones’ self was expressed in the interviews mid-way through and at the end of the semester. These necessary objects discovered in the interviews may support the theory of constructivism. Self-regulation, increasing difficulty, and change in modality (De Corte 2003) all promote learning that is encouraging students to create their own knowledge. Active and dynamic learning is an essential goal for business students learning management information systems. Many of them will be using management information systems exclusively in their initial jobs.

Some students expressed a desire to replace the online activities with face-to-face classes, confirming previous studies (Owston et al. 2013; Farley et al. 2011). Negative pressures are displayed in the learning mechanism (Fig. 2): questioning the professor or fellow students is easier in a face-to-face environment. And many students felt that there were too many distractions online; they could focus better in a face-to-face situation. This was especially evident towards the end of the semester when the class became more complex.

Fig. 2
figure 2

Learning mechanism

Full size image

An upperclassman (junior) with a GPA of 3.261 began the blended learning class by explaining that he liked the idea of the class modality and that he only gets “so much” out of lectures. By the end of the class, he was saying that he was very disciplined, knows how to get the work done, and how the class worked out very well for him. These comments concur with the results from Farley et al. (2011) in that the upperclassmen embraced blended learning more readily than the lowerclassman, and that of Owston et al. (2013) in that high achievers are more satisfied with blended learning. In a 2013 editorial, Owston stated “academic ability is a critical factor in determining success of the BL student.” (Owston 2013).

There were other lessons learned from this experimental class. Blended learning has allowed for adding extra material outside of face-to-face classroom time, akin to supplemental instruction sometimes used in tutoring scenarios or study sessions. It appears that the content of what is ported online is important to the students. This idea of certain material being better suited to online work is one worth noting. One student claimed “the stuff that we do online … you can just read the book and learn it so I don’t think it would matter if she taught it to me in person or I read the book but I think it depends on the material so it depends on what the class [is]… but other classes it might make a difference This may be an important reference point for future classes, especially those in STEM, considering blended learning.

Apart from the academic learning, as with previous studies focused on hybrid courses at this university, students in this course were very happy with the flexibility that this new course schedule allowed them. Although some students appreciated the asynchronous online component simply because it allowed them to sleep later on Friday, others used this flexibility to better accommodate their busy schedules, such as sporting activities. Students have also mentioned that faculty availability is essential for operating a class in this mode. Although not all students take advantage of class time and faculty office hours to ask questions or seek help, quick responses to e-mail messages have mitigated the decrease in face-to-face contact. These comments concur with past research in accounting and finance where students claimed that the online material should be well-organized and applicable (Osgerby 2013).

The pedagogical implications from this research are as follows. From the statistical results, it appears that replacing 1/3 of the traditional face-to-face class time with online activities does not affect the final grade outcomes of this MIS class. Students, although preferring face-to-face class time which have little or no distraction, were accepting of the online activities. However, those online activities and material had to be well-organized and relevant to the class. It was clear from the interviews that students began to think more about time management and some indicated their enthusiasm for self-learning. As the business world becomes more complex and competitive, any encouragement of autonomy and self-learning is beneficial to our students’ future careers.

5 Limitations and future work

There are several significant limitations of this study. First, one of the research questions focused specifically on international students but very few international students enrolled in this course during the one semester in which a blended mode was piloted. None of them volunteered for interviews and their very low numbers raises concerns about the generalizability of the statistical analyses even within similar contexts e.g., MIS courses. Second, only one professor conducted her classes in a blended mode in one semester. A more expansive, generalizable experiment would have multiple professors running this management information systems class in a hybrid fashion. In hopes of creating a rigorous experiment within the limitations and resources available, the blended class was conducted in a very similar fashion to the previous semester’s traditional class. Content stayed exactly the same except that 1/3 of it was ported online. As the results show from the qualitative data, the online portion did not connect well with the in-class portion. Future blended classes will improve upon this model and do a better job in connecting both segments of the class. That we could only utilize the final grade in the course as our measure of student learning is also a limitation. This single measure lacks nuance and is idiosyncratic to this one course at this one university. As universities move more class time to an online format, more data can be collected and analyzed to further refine pedagogical direction. Finally, only students from the blended learning class were interviewed. The research was conducted after the face-to-face class had ended, so this comparison of blended versus face-to-face was not possible.

6 Conclusion

Universities are rethinking the way they deliver courses and using today’s technology to offer blended education with a combination of face-to-face lectures and online material. Recent research on blended learning has shown the potential for this modality to yield increased exam scores or positive feelings, however many of these research studies are based on totally redesigned courses, or courses that have simply added online material. This research addressed the limitations of many past studies by assessing learning for a business information systems class that was conducted in a blended mode. This blended course held the course material and professor constant, porting one-third of the classes to online. Using a mixed method of data collection, final grades and interviews, the outcomes of this blended learning class were compared to nearly identical courses taught in the immediate past, making this a unique study. Through a critical realism lens, the data were analyzed quantitatively and qualitatively to create a mechanism for learning.

The goal of this research project was to examine how well the students met the learning outcomes from a blended learning curriculum and to develop a model for blended learning. By answering the research questions posed, (1) what are the changed outcomes to a blended learning class when the content is the same as a face-to-face class; (2) is blended learning a superior learning environment; and (3) is there a significant difference for non-mother tongue speakers with blended learning, the goal has been met by collecting quantitative and qualitative data and analyzing the data by multiple regression models and content analysis. The results of this analysis show that blended learning is comparable to face-to-face learning in terms of grades, although students prefer face-to-face learning. Disappointingly, there were few international blended learning students in this study (5). For those students the difference in grades was not large but statistically significant with lower blended learning scores.

These research results, in some way, validate the efforts of many universities who teach management information systems in a blended mode. Suggestions for blended learning educators such as keeping the level of online class to 1/3, ensuring that the online material is well-organized and relevant, and encouraging self-learning and time management, should assist other schools that wish to teach in this modality.