Introduction

Over the past decade, calls for reform in medical education have intensified, each citing evidence that traditional pre-clinical medical education has become too dependent on lectures and other passive didactic models and that these methods are less effective in equipping students with knowledge and problem solving skills for clinical contexts [1–5]. Several of these articles have proposed implementation of new teaching methods, such as the flipped classroom, to enhance pre-clinical medical education. These models of classroom instruction rely primarily on student preparation outside of class in order to use in-class time for specific kinds of active learning activities, such as problem-based learning (PBL) or team-based learning (TBL) [6–8]. The flipped classroom model denotes a slightly different approach to in-class active learning, where students are responsible for learning the basic concepts on their own, usually through online videos, and classroom time is utilized in a wide variety of active learning activities [9, 10].

Blended learning approaches, like the flipped classroom, that utilize online technology and instructor-led active learning experiences have shown promising results in several studies. For instance, a large meta-analysis of online and face-to-face blended learning studies demonstrated that while purely online instruction leads to similar results as in-class instruction, blending online and in-class instruction shows a significant improvement in educational outcomes [11]. However, these results were based on predominantly undergraduate and other graduate health professions courses that did not include pre-clinical medical education. Further, while the flipped classroom approach of blending learning has continued to show significant improvements in student performance in undergraduate [12], nursing [13], graduate school [14], and pharmacy [15] courses, few studies have been published on the success of specifically named “flipped classrooms” in preclinical medical education [10, 16]. Still, it is difficult to assess the exact number of published studies due to ambiguous terminology used to describe blended learning classrooms, such as the flipped classroom.

While it is likely that the success of the flipped classroom should extend to pre-clinical medical education, medical students and medical curricula face some obstacles that other curricula may not. For example, pre-clinical medical education is already challenged to expose students to a large amount of information in a very short time, and students spend large amounts of their time outside of class studying to stay caught up with material [17]. Thus, putting additional requirements for students to acquire information on their own outside of class could further impair student mental health and well-being [10]. Further, preparing students for board examinations requires that all pertinent information must remain in the curriculum, leaving little room to cut material in order to create extra time for more in-depth analysis and application. Therefore, it is necessary that the efficacy of the flipped classroom be studied in a pre-clinical medical school curriculum to determine if the successes demonstrated with other health professions transfer to medical education.

Another important question in flipped classroom implementation is which methodologies work best for the outside-of-class and in-class learning experiences. Some articles have suggested that online videos resembling the Khan Academy model could be used as a way for students to learn the information [10], but there are other applications of technology that could also prove beneficial such as online textbooks or interactive modules [18, 19]. In-class active learning experiences could take many forms including case method teaching [20], simulation [21, 22], peer instruction or think-pair-share questions [23, 24], audience response questions [25], collaborative learning [26], or in-class patient presentations. However, is it possible to implement these experiences in a way that adds value to the core material without adding additional time that takes away from much-needed personal study time? Again, these questions argue that implementation of the flipped classroom must be studied directly in pre-clinical medical curricula in order to determine the best approach.

In an attempt to answer several of these outstanding questions, we implemented a fully flipped curriculum in a pre-clinical basic science course. Our primary goal was to determine if a fully flipped basic-science course could improve student exam performance and overall satisfaction, while at the same time being time-neutral for student study time requirements. Further, we wanted to experiment with several approaches to the delivery of didactic material as well as active learning exercises to ascertain if students preferred any particular method.

Methods

Course Description

“Integrative Function and Its Cellular Basis” is a required, 10-week, basic-science course in the first year curriculum that covers basic (normal) neuroscience, neuroanatomy, histology, and systems physiology. For the 2013–2014 entering class, the first 4 weeks, composed of neurophysiology and neuroanatomy, were taught using traditional lecture-based approach, and the last 6 weeks of the course, composed of cardiovascular, respiratory, renal, gastrointestinal, and reproductive physiology content, were taught completely by the flipped classroom approach. For these final 6 weeks, lectures were replaced with online modules that consisted of several short videos, and one to three quiz questions that followed each individual video. Videos ranged from 1.5 to 17 min for longer topics, with an average video time of 7.5 min each. The videos predominantly covered foundational material that was mostly void of clinical application and integration. For some topics (pharmacology, reproductive, and gastrointestinal physiology), lecture capture from the previous year was edited into short videos that included the pertinent information. For the remaining topics (cardiovascular, respiratory, and renal physiology), new videos were generated using the screen capture feature of Adobe Captivate software, Microsoft PowerPoint slides, and voice-over instruction. The individual videos and quiz slides were then complied into modules using Adobe Captivate software and uploaded to the course Sakai website, along with lecture syllabi and the Microsoft PowerPoint files used to create the videos.

In place of lectures, we introduced interactive sessions called “Drilling on Concepts” or “DOC” sessions and continued the small-group activities that were used in previous years. Each week included at least one DOC session and a small-group activity. The DOC sessions were 1- to 2-h, instructor-led sessions that gave students opportunities to apply and integrate the foundational concepts taught in the modules. A variety of active learning approaches were used to review and reinforce the concepts taught in the online modules (see Table 1). For instance, the cardiovascular DOC sessions utilized simulation technology followed by a small-group session that reviewed pertinent physiology using audience response system (ARS) questions. For other DOC session, students were given short cases to read, followed by audience response questions that encouraged small-group discussion. Further, real-patient and mock-patient sessions were used to incite students to apply physiology to clinical scenarios (see Table 1 for more description). Students met in smaller groups of 36 for the DOC sessions, and within these groups of 36, smaller groups of 6 were formed to encourage discussion of the ARS questions. Thus, DOC session activities were aimed at clarifying and applying clinically relevant and difficult concepts from the modules, as well as integrating concepts that apply to the organ system as a whole. When necessary, instructors taught using Socratic method during DOC session discussions in order to clarify concepts, and answer student questions. Instructors mainly addressed audience response questions where the real-time feedback suggested that a large percentage of students were missing important concepts. Further, they also addressed ways in which students could integrate and apply specific concepts when it was clear that students were not able to do this on their own. Additionally, similar to previous iterations of the course, each week was concluded with a small-group activity, in which groups of students worked through application exercises with a faculty member present to facilitate the process as needed.

Table 1 Active learning activities utilized for each unit of study
Full size table

In order to increase faculty accessibility, we established extended office hours. In the past, the course director had office hours that stretched throughout the afternoon on all days of the week; however, during the flipped classroom, the course director was also available during morning hours when no other active learning sessions were scheduled (usually 1–2 days a week for 4 h each day). Further, an optional daily review session, led by the course director, was offered to students from 1:00 p.m. to 2:00 p.m. This review session was open for students to ask questions to the course director specifically regarding the module that had been assigned the previous day. The instructor used a white board rather than powerpoints to address questions. If there were not enough questions to fill up the hour, the instructor would informally go over several difficult concepts covered in the module. However, these review sessions were not lectures in that they were not prepared beforehand, did not utilize PowerPoint slides, did not cover all the material from the modules, and often relied on students to explain concepts to other students when possible. Less than 20 % of the class routinely participated in these review sessions.

Sample

Two cohorts of students were compared in this study. The 2012–2013 cohort (group 1) consisted of 180 students, 45 % of which were female and 55 % male. North Carolina residents made up 84.4 % of the students, and 66 % of the students considered themselves white, while 12 % identified as African-American, 1 % Native American, 13 % Asian, and 8 % did not report a racial identity. The 2013–2014 cohort (group 2) consisted of 180 students, 52.8 % female and 47.2 % male. North Carolina residents made up 87.8 % of the students, with 57 % of students identifying as white, 13 % as African American, 18 % as Asian, and 12 % choosing not to identify.

Instrumentation

Examinations

A total of 191 common items were administered to students across both cohorts. Of the 191 items, 80 items were considered control items as they were based on content that was delivered to both cohorts via lecture in a virtually identical manner.

The remaining 111 items were considered treatment items as they were based on content that was delivered in different instructional formats, particularly lecture (for group 1) versus the flipped classroom (for group 2), across the 2 years. The purpose of this design was to determine the extent to which the two cohorts performed similarly on common items to establish a frame of reference. If evidence of comparable performance on control items was available, any potential differences resulting from the treatment items could provide some evidence of learning that speaks to the effectiveness of the instructional modality.

Course Evaluations

The course evaluation administered to both cohorts consisted of 26 common items. A total of 18 items evaluated aspects of the course that were common to both instructional formats; the remaining 8 items evaluated relevant aspects of the course (e.g., instructional practices, content delivery, and course quality) that contrasted the traditional versus flipped classroom experience.

Flipped Classroom Survey

A survey specific to aspects of the flipped classroom was administered to the 2013–2014 cohort. This survey contained 46 items that were specific to features of the flipped classroom. This survey was designed by assessment staff in the Office of Medical Education (OME) with input from the course director. Students in the 2013–2014 cohort were asked to subjectively compare their experience in the flipped classroom to their experience in the first 4 weeks of the course in the traditional lecture, since they had not experienced a lecture-based approach for material in the last 6 weeks of the course. Therefore, the results of this survey are purely subjective student responses of how they felt the flipped classroom compared to the traditional classroom overall. However, it should also be noted that both the first 4 weeks and the last 6 weeks of the course covered the broad topic of physiology, enabling students to at least compare approaches within the same broad discipline rather than across disciplines (e.g., anatomy or biochemistry vs physiology).

Data Collection

Student examination and course ratings data were collected independently by the OME. The office uses a sophisticated online testing software program named MedSTARS to administer all of the school’s examinations and quizzes, and the One45 software program to collect student course evaluation ratings.

Data Analysis

We used a quasi-experimental design to compare students’ examination performance and course evaluation ratings across the two cohorts. Before any inferential analyses were conducted, we performed a series of rigorous psychometric analyses using item response theory modeling to investigate item quality and functioning, and score reproducibility. These “validity checks” provided sufficient evidence to move forward with various inferential analyses. Descriptive statistics were produced to discern score distributions, variation, and score differences. Independent samples t tests were used to compare mean scores across groups to detect statistically significant differences. Cohen’s d effect size estimates were also generated for items that were flagged as statistically significant so as to understand the magnitude of the “practical significance” of each difference. This study was approved by the University of North Carolina Institutional Review Board (#14-1218).

Results

Impact on Instructional Time

One of the primary goals of this flipped classroom implementation was to be time-neutral with regards to the out-of-class and in-class time required of students so as to not increase student time commitments over and above what was required in the traditional (lecture-based) classroom. In order to estimate the time required to view material during the first pass (assumed to be the equivalent of sitting in a lecture), we added up the time spent in lecture in the 2013 version of the course, and the total time it took to watch the modules, at a normal speed. We also added in 2 min for each video for the student to answer the quiz question at the end. Using this methodology to estimate the time required for students to acquire the foundational information on the first pass (via either lectures or modules), we calculated that the video modules potentially reduced the instructional time spent on foundational principles (see Table 2). Active learning activities were added based on the time saved by the video modules in order to keep the total instructional time equal to or less than instructional time used in previous iterations of the course (Table 2). Thus, students continued to spend approximately the same or less time in formal instruction (gray columns denoting “Total instructional hours” in Table 2). However, this approach is merely an approximation to total time spent by the students, since in a flipped classroom, students are expected to come prepared for class, which goes beyond merely watching the videos. In addition, students must take time to process and digest the information in order to be ready to participate in classroom activities.

Table 2 Hours of instructional time compared between traditional classroom (2013) and blended classroom (2014)
Full size table

To determine if our estimates of total required student time (modules + interactive activities) were perceived by the students to be equal to the time used in the traditional classroom (lecture + small group), we asked the students to subjectively asses if they spent more time studying in a traditional classroom, the flipped classroom, or about the same in each. Twenty-eight percent of the students responding to the survey (N = 143) said that they spent about the same amount of time studying during the flipped classroom compared to the traditional classroom, while 37 % reported spending more time in the traditional classroom, and 35 % reported that they spent more time studying with the flipped classroom. Further, to assess whether we had struck the right balance with individual study time and in-class instructional time, we asked the students if they felt that the hours dedicated to individual study time were too few, to many, or appropriate. A total of 143 students responded, resulting in a 3.73 % margin of error with 95 % confidence. The majority of the students (83.4 %) reported that the amount of course time dedicated to individual study was appropriate, with only a small percentage reporting that it was too little (5.0 %) or too much (5.7 %). Likewise, when asked to subjectively evaluate the hours dedicated to active learning experiences (small groups and DOC sessions), the majority (86.6 %) agreed that the number of in-class experiences was appropriate, with only a small minority reporting that there were too few (4.9 %) or too many (8.5 %).

Impact on Student Performance

We compared students’ performance on both control and treatment items to discern if any differences were likely to be authentic. We began by examining students’ performance on the 80 common items in which instruction across both groups was based solely on lecture format. This comparison of control items allowed us to determine if the items were functioning differently across groups and if the students demonstrated unbalanced levels of performance from the start. An independent samples t test indicated that matched item p values were not statistically significantly different (p = .213) with a 95 % confidence level. With regard to students’ performance on the control items, there were also no significant differences evidenced between the groups (see Table 3).

Table 3 Students’ performance on control items
Full size table

Next, we examined students’ performance on the 111 treatment items. That is, students from the 2012–2013 cohort had received only traditional lecture-based instruction and were provided these 111 items; students from the 2013–2014 cohort, however, had received instruction based on the flipped classroom model and were also provided these same items. Results indicated that there were significant differences in student performance (see Table 4).

Table 4 Students’ performance on treatment items
Full size table

When comparing the two cohorts of students’ performance on the items that were administered via different instructional mediums, students’ performance emanating from the flipped classroom resulted in slightly higher scores (about 2 points on average). An independent samples t test indicated that the percent correct values across the 2 years were statistically significantly different, t(355) = 2.238, p = .026, with alpha at .05 confidence. The Cohen’s d effect size estimate of .237 indicated that the practical significance of the score differences was small to medium in magnitude [27]. The p value of .053 indicates that the average student scores barely fell outside the range of statistical significance; however, the effect size estimate of .20 indicated a small to medium practical significance.

Students’ Ratings on Course Evaluations

We compared students’ responses on the course evaluations across the 2 years. Although the course evaluation instrument contained a total of 26 items, we opted to investigate only the items (n = 8) that pertained to instructional practices, content delivery, and course quality due to relevance (see Table 5). Questions excluded from the table involved topics such as clearly defined course and learning objectives, fair grading practices, and course syllabus, which were not the focus of this particular educational intervention. Of the eight relevant items evaluated, five were statistically significant with alpha at .05. The Cohen’s d effect size for the statistically significant items ranged from medium to large in magnitude, thus indicating a notable practical significance.

Table 5 Students’ ratings on course evaluations
Full size table

Students’ Perceived Benefits of the Flipped Classroom

The course evaluation data suggest that students perceived the course more favorably than in previous years, perhaps due to the addition of blended learning in the form of a flipped classroom. However, since a portion of course also involved traditional lecture-based instruction, we cannot rule out the possibility that student favorability was due in part to the lecture portion of the course and not the flipped classroom. An additional survey given to the students to investigate their perception of the flipped classroom asked students to subjectively rate which method (flipped or traditional classroom) aided their ability to master material in several different ways (Fig. 1). These data represent subjective student feelings regarding comparisons between flipped and traditional classrooms since the students participating in the survey had not previously had the specific organ physiology (cardio, renal, respiratory, etc.) taught in a traditional classroom. Instead, they were asked to compare their overall approach and mastery of material in the course with either a flipped or a traditional classroom. A total of 143 students responded, resulting in a 3.73 % margin of error with 95 % confidence. In most categories queried, more students preferred the flipped classroom compared to students that preferred lecture, with a very strong contingency of students reporting no preference for either flipped or traditional classrooms. Interestingly, a majority of students felt that the flipped classroom better helped them increase their analytical thinking (58.5 %) and problem solving skills (61.5 %) compared to the traditional classroom.

Fig. 1
figure 1

Students rank flipped classroom higher than traditional lecture in several different assessments of learning. Histogram showing the percentage of students who preferred lecture-based curriculum (dark gray), flipped curriculum (medium gray), or had no preference (light gray), in six different types of learning. The numbers within each bar denote the actual number of students each bar represents

Full size image

Discussion

Taken together, these results suggest that the flipped classroom can be successfully implemented in pre-clinical medical education. Our implementation of the flipped classroom focused on the total in-class and out-of-class time commitment needed by medical students, so as to not increase the already large amounts of time required by students to learn and understand the material. By replacing lectures with short videos that focused only on the fundamental principles of medical physiology, and saving the application and integration of this material for in-class, interactive sessions, we were able to create a seemingly time-neutral change from a traditional classroom to a flipped classroom. While many studies have discussed the benefits of the flipped classroom, often they reference the increased burden incurred on the students from having vast amounts of information be transferred to out-of-class hours [15, 4]. Students in our flipped classroom reported that this implementation had hit a good balance of both in-class activities and out-of-class personal study time to acquire the information.

Further, student performance and satisfaction with the course improved with the addition of the flipped classroom, reinforcing previous studies done in other health professional schools [15]. These results suggest that the flipped classroom is a viable option for pre-clinical medical education. Thus, these results are in line with other studies that have reported favorable outcomes in student performance and satisfaction using other methods that also utilize a flipped classroom approach, such as TBL, PBL, and e-learning [18, 28–32]. Thus, our study reinforces these earlier studies that demonstrated that the flipped classroom-like approaches work well in pre-clinical medical education.

Successful Aspects of the Flipped Classroom

While the data show that the flipped classroom resulted in a small improvement in student performance and satisfaction, student comments in the course evaluations suggested that specific aspects of this implementation were necessary for its success. First, it is clear that students appreciated the variety and quality of both the study materials used to learn the material, and the in-class experiences used to reinforce and apply essential physiological concepts. Thus, it is important to provide several types of personal study materials to give students the option that meets their learning preferences. Further, it was clear from the comments that the DOC sessions were very important for students to practice applying the material learned via the modules/syllabi. The DOC sessions were nearly universally praised as being high quality, and essential for integrating difficult concepts. Students also appreciated the variety of activities that were utilized in the DOC sessions. Each one of the DOC session methods received at least one comment by a student that it was their favorite DOC session. Thus, since the flipped classroom opens up class time to utilize many active learning approaches, it makes sense to not rely on any single approach, but to use a variety so as to engage as many students as possible throughout the course. Others who have implemented the flipped classroom have also pointed out the importance of variety [16].

Students also commented extensively on the quality and focused nature of the videos produced. In general, the students tended to prefer the videos made specifically for the flipped classroom over the videos that were made by editing the prior year’s lectures. Although, this preference for screen captured, voice-over videos was not universal, as a minority of students did comment that they preferred videos that were directly made from a person giving a lecture to an actual audience.

Other Points to Consider

Despite the seeming success of the flipped classroom in this pre-clinical physiology course, several remaining issues became apparent in the midst of the implementation and from student comments in the course evaluation. It is likely that these issues are applicable to medical students at other universities and so we will discuss them here.

First, it is apparent that between 10 and 15 % of the students strongly prefer a lecture-based curriculum over the flipped classroom. Even with the extensive faculty availability and the flexibility the flipped classroom affords students, a small but strong contingent of the class was very unhappy with this new approach to medical education. Both the demographics of the UNC School of Medicine student body and the comments in the course evaluation shed some light onto why this strong minority struggled to adjust. The majority of medical students matriculating at UNC consist of students who come from large state schools where lecture is the predominant form of teaching (64 % of the 2013–2014 cohort). Thus, it is likely that a flipped classroom deviates considerably from what UNC medical students have become accustomed to. Students commented that they were very apprehensive to lose lecture as the primary form of learning, despite having videos that replaced it, and several students commented that they would have not chosen to come to UNC if the flipped classroom was a large part of the curriculum. UNC undergraduate campus, where 34 % of students in the 2013–2014 cohort received their bachelor degrees, is in the process of wide implementation of the flipped classroom in several large courses across campus. It will be interesting to see whether the attitude of this small contingent of students changes as more students are exposed to the flipped classroom earlier in their education.

Further, a majority of students commented that while they enjoyed the flipped classroom, they missed having daily interactions with their peers. In fact, only 10.5 % of students reported that they did not miss their peers at all, and 38.4 % of students reported that they missed seeing their peers a great deal. Additionally, the most common response to the survey query “What if anything, from the lecture-based curriculum did you miss in the flipped classroom” usually expressed the theme of seeing peers or having a sense of community created by attending lectures together. When designing our flipped curriculum, we anticipated the potential for students to feel cut off from their social group, so we reserved rooms with large screens and computers so that students could watch modules together if they desired. However, the majority of students reported either never watching modules with friends, or only watching with friends on a few occasions. These results are consistent with a previous study that reported only a minority of medical students study in groups [33]. It is possible that having more large-group interactive activities would enable the lecture community feeling to be a part of a flipped classroom. However, interactive activities with 180 students in one large room present problems as well, such as students not feeling comfortable asking questions. Approaches to the flipped classroom at other medical schools might find innovative ways to salvage this sense of community at the same time promoting independent learning. However, for now, this issue has yet to have been solved.

Further, it is also clear that some do not consider online videos an equivalent to a live lecture, even though studies have shown that outcomes vary little between online and live instruction [34, 35]. Comments by several students expressed preferences for lecture over the flipped classroom that centered around missing the theater of lecture, i.e., the stories that lecturers tell as they are presenting the material [36–38]. Other students also commented that they missed being able to ask questions of the teacher in real time during the modules. Some students got around this by either sending emails, or watching modules during office hours at school. However, some students also commented that it is not only the questions that they themselves ask during lecture that are beneficial, but also questions asked by their classmates. Many students suggested an online forum where all questions and answers would be viewable by all students. This is a simple suggestion that could easily be implemented enabling students to feel more like they were part of a community. In fact, other independent-learning courses implement this type of forum with successful results [39].

Finally, the work effort on the part of the faculty to implement the flipped classroom successfuly must be addressed. As previously discussed in this article, and in other articles [15], extensive office hours and review sessions were made available to students so they would have the support they needed during the flipped classroom. Students frequently commented that the success of the flipped classroom in this instance was only due to the effort of the course director in creating a community and by offering office hours every day of the week. Students were adamant that had any less effort been put forth by the course leadership, the flipped classroom would have been less well received. In fact, it is possible that these additional office hours and review sessions contributed in part to the overall success of the flipped classroom in this study. Thus, when planning flipped classrooms, it is important to realize that more effort is needed on the part of the faculty.

Study Limitations

While these results argue in favor of the flipped classroom over traditional approaches, there were at least two limitations in our study design. First, even though student participation was high in the flipped classroom survey (143/180), we cannot rule out the potential for non-response bias. Second, we collected data from only two cohorts, one from a traditional classroom and one from a flipped classroom. While the demographics of the two cohorts are similar, we cannot necessarily extrapolate our findings to pre-clinical classrooms in which the student body differs considerably with regard to demographic composition. Further, the 2013–2014 could have been biased by knowing the intent of the flipped classroom evaluation since they knew that this was the first time the intervention had been tried at UNC School of Medicine. Future research should focus on comparisons across multiple cohorts and student samples consisting of different demographic compositions. The collective body of research will continue to shed light on the feasibility and success of the flipped classroom in medical education.

Conclusion

In conclusion, our implementation of the flipped classroom in a pre-clinical physiology course was successful based on student academic performance measures, course evaluation and instructional preference ratings, and qualitative feedback. However, there are still outstanding issues to be resolved before all medical students and faculty wholeheartedly embrace this approach to medical education.