As online and hybrid courses flourish within universities (Allen and Seaman 2015), digital video is an increasingly popular way to engage millennials in online learning (Project Tomorrow 2015). One powerful form of educational video is digital storytelling, short video vignettes that incorporate imagery, data, and narrative stories to teach complex content in an approachable way (Dreon et al. 2011; Kajder and Swenson 2004; Lambert 2013; Robin 2006, 2008; Spierling et al. 2002).

The current study explores digital storytelling within a hybrid course on sustainability science for preservice teachers. Within the online portion of the course, digital storytelling is used to introduce the complex ideas of sustainability science to a lay audience (Warren et al. 2014). It also ensures a consistent content presentation across instructors, while being accessible to students in a format that supports self-paced learning. Nonetheless, the use of online digital storytelling video poses tensions that are historically well-documented issues in teaching and learning, including how to: promote student engagement (Shephard 2003), scaffold learning (Hannafin 1985), hold students accountable (O’Flaherty and Phillips 2015), as well as the issue of content retention. We examined the use of interactive quiz questions embedded within digital stories as a way to mitigate some of these tensions. Practical applications for the use of interactive digital storytelling within online and hybrid university courses are discussed.

Learning Through Digital Storytelling

Digital storytelling is a form of video communication that incorporates images and sounds along with narrative to address a perspective or story (Robin 2006, 2008). Digital stories differ from mainstream video in that they serve an educational purpose, can be produced simply and affordably by instructors and students, and tell a unique set of stories, narrated by people connected to the story itself. They were first popularized within educational settings in the 1990’s, as accessibility to image and audio capturing devices and digital media software widened (Lambert 2013). While research on this topic has primarily focused on student-created digital stories in K-12 settings (Kajder and Swenson 2004; Dreon et al. 2011; Hung et al. 2012; Yoon 2013) as well as higher education (Lowenthal and Dunlap 2010; Rambe and Mlambo 2014), the current study focused on students’ experiences with instructor-created digital stories.

In this study, digital storytelling video was created by an interdisciplinary team of content experts, researchers, educators, and video/graphic designers, for use in the online portion of a 15-week hybrid course, Sustainability Science for Teachers (SSFT). Hundreds of undergraduate education majors take this required course each semester, in course sections of 20 – 35 students. Each week, outside of class time, students watched approximately 1 h of online video on their personal computing devices. All video was housed within the learning management system (LMS) for the course. These digital stories incorporated gripping case studies in a format inspired by the Public Broadcasting Service’s NOVA program to teach sustainability topics such as population, poverty, food, water, and fossil fuels. Then, students met in an hour-long face-to-face class (FTF) with 20 – 35 other students once a week. The FTF time engaged students with related hands-on activities aimed at fostering pedagogical content knowledge (Shulman 1986) and supporting the integration of the sustainability science into their future teaching. Digital storytelling video was used as the principal content for the online portion because of the following practical advantages, it: (a) could be produced simply and affordably, (b) capitalized on unique sources of knowledge from contributing researchers and educators, (c) was readily available to students with online access, (d) could be archived and used across semesters and available to students in the future, (e) ensured consistent content presentation across course instructors, and (f) resonated with the preservice teacher audience, as digital storytelling is a common educational tool.

Digital storytelling also offers pedagogical advantages. First, because our student population began the course with little understanding of sustainability (Foley et al. 2015), the complex topics of sustainability could be conveyed more holistically via video than with technical articles or textbooks. The stories captured that which cannot be best shared in print. Second, when teaching sustainability, incorporating footage from across the world was important to conveying the depth of the course topics. Mar and Oatley (2008) posited that a unique advantage of video is the ability to provide students with vicarious experiences that exceed the range of experiences possible in real life. Third, digital storytelling connects emotionally with students. As LaMarre and Landreville (2009) demonstrated, informative films that produce affective responses were also associated with increased topic interest and narrative engagement. Finally, digital stories seek to promote learning and retention by providing a rich stream of information through multiple modalities. Evidence suggests that video offers learning benefits as compared to more one-dimensional forms of content, because it incorporates visuals, moving images, and sound in a cohesive track (Burmark 2004; Hibbing and Rakin-Erikson 2003).

Interactive Digital Storytelling

A challenge of using storytelling video as the principal course content is how to make learning less passive and more interactive. Those investigating learning with interactive video have each defined interactivity somewhat differently (Cherrett et al. 2009; Delen et al. 2014; Merkt and Schwan 2014; Merkt et al. 2011; Zhang et al. 2006). Looking at the overall body of work, interactive video features may include the ability to:

  • Re-view or skip video chapters and content

  • Pause to process content

  • Use electronic indices to support further inquiry

  • Answer electronic pop-up questions to direct focus and/or check for understanding

  • Prompt the student to select what information is to be learned next (Schwan & Riempp 2004)

For the current study, we define interactive digital storytelling as video that includes scaffolding features to prompt students to actively engage with the content, and is designed to tailor the learning experience to individual students’ needs.

A number of experimental studies have examined interactive video features’ relationship to learning outcomes. Delen et al. (2014) demonstrated learning gains when students were able to select video chapters to skip or re-watch, while Merkt and colleagues (2011) demonstrated learning gains when electronic indexes were made available to support further investigation of content in the video. Similarly, Zhang et al. (2006) demonstrated learning gains when content was presented in multiple formats on one screen. Alternatively, Merkt and Schwan (2014) found interactivity within video content did not always support student learning. While an educational video may be designed with optimal interactive affordances, the degree to which an individual learner might interact varies (Downes and McMillan 2000; Hannafin 1985; Merkt and Schwan 2014; Rafaeli 1988; Turrión 2014).

The current study addresses two gaps within the literature. First, most of the experimental studies of learning with interactive video have explored a relatively decontextualized one-time snapshot of students’ retention after viewing the material (Cherrett et al. 2009; Delen et al. 2014; Merkt and Schwan 2014; Merkt et al. 2011; Zhang et al. 2006). Our study examines interactive digital storytelling within an authentic 15-week course, implemented on a weekly basis. Second, we explore the issue of pedagogical integrity, or ensuring that students engage with online video in the manner intended by the course’s learning objectives. In their review of hybrid learning in higher education, O’Flaherty and Phillips (2015) explained that pedagogical integrity is one of the most critical challenges threatening the success of hybrid classroom approaches. They found that, “Students are less likely to engage in pre-class activities that lack interactivity, fail to provide formative feedback mechanisms, and are/or not coherently linked to the face-to-face class” (O’Flaherty and Phillips 2015, p. 94).

Prior to this study, pedagogical integrity had become a legitimate concern within the SSFT course, as instructors learned that students could avoid viewing the digital stories yet still score well on the post-video content quizzes. Post-quizzes were used as an accountability measure, completed in addition to homework activities such as written reflections and lesson planning that challenged students to analyze and apply the content. To address instructors’ concerns while keeping the post-quiz assessment format, the existing storytelling video was modified so it would pause after important information, requiring a student’s response to advance. Students were exposed to both interactive and conventional digital stories.

The purpose of this study was to examine the use of interactive digital storytelling video for increasing student engagement, scaffolding their learning, promoting learning gains, and holding students accountable for viewing the online content. We surveyed students at the end of the semester, addressing the following research questions:

  • What are students’ perceptions of learning with interactive, as compared to conventional, digital storytelling? Of specific focus were the themes of student engagement, scaffolded learning, learning gains, and accountability.

  • What unpredicted disadvantages do students identify with interactive digital storytelling?

We also analyzed students’ quiz performance when using an interactive format compared to the conventional digital storytelling video format within subjects, to address a third research question:

  • Does the interactive video format, when compared to the conventional format, increase student learning gains?

Method

We employed a survey research design (Czja and Blair 2005), with a mixed methods approach (Creswell 2015). Participants included two cohorts of preservice teachers enrolled in SSFT, over two concurrent semesters, at a large public university in the Southwestern United States. Each week in the course, students watched online digital stories comprised of a series of five to 15-min segments, lasting between 44 and 96-min in length (M = 62 min). These digital stories were accompanied by multiple-choice quiz questions. While watching, students had complete control over starting, stopping, pausing, rewinding, and fast-forwarding. They were able to watch and re-watch the video as many times as they wished. For approximately half of the semester, students were exposed to interactive digital storytelling video, and for the other half they were exposed to conventional video format. The only way the two formats differed was that with the interactive format, the video was programmed to pause with a quiz question on the screen after critical information was presented. This had to be answered prior to advancing. There was no time limit for responding. At the end of the video, students reviewed their answers for final approval. This was achieved by integrating the interactive video tool, HapYak, which allowed the quiz questions to be inserted into the existing video so student response data could be collected. In the conventional format, students watched the same video without the embedded quiz questions and pauses. At the conclusion of the video, they answered the same quiz questions, which were presented within their LMS. Across both formats, the structure and content of quiz questions were consistent, and students were able to access the questions prior to the quiz as a learning guide throughout the video. All students were exposed to both formats for approximately half of the semester.

At the conclusion of the course, the 398 enrolled students were surveyed to explore their experiences with the digital storytelling formats. A total of 223 participants submitted surveys (n = 147 Semester A; n = 76 Semester B), indicating a response rate of 56 %. Based on the 220 participants who provided demographic information, the sample consisted of 194 females (88 %), and included students who self-identified as white (n = 146), Hispanic/Latino (n = 46), American Indian (n = 9), Asian (n = 9), Black or African American (n = 4), or two or more races (n = 4). All participants were undergraduate Kindergarten – 8th grade preservice teachers.

Survey Instrument

The web-based survey was developed by a sustainability expert, a pedagogical expert, and the graphic designer for the course. An iterative process was used to design and review the survey (Czja and Blair 2005). It included 18 Likert items, which explored perceptions about the helpfulness of the two video formats, across nine survey items (18 items total), using a 4-point Likert Scale including “1” (Not at all), “2” (Very little), “3” (Some) to “4” (Quite a bit). Two open-ended questions regarding perceived advantages and disadvantages of the interactive digital stories followed.

Data Collection and Analysis

Analyses employed a comparative mixed methods approach (Creswell 2015). Quantitative data were analyzed using descriptive and inferential statistics, and qualitative data were open coded, drawing on a grounded theory approach (Dey 1999). Qualitative analyses began with a first round of coding, in which all student responses were read and reoccurring themes were identified. Then, the research team discussed recurring themes until a consensus was reached regarding the codebook. A final round of coding followed, when all participants’ responses were re-coded using the finalized set of themes. A single participant’s response could be coded for multiple themes.

Limitations

One limitation of this study — and survey-based research in general — is that participants’ self-reported beliefs are not necessarily reflective of their actions or observable experiences (Fowler 2002). In this study, we present an analysis of students’ quiz scores, but results should be interpreted cautiously because the quizzes were designed as understanding checks, so most students scored well regardless of video format. Similarly, non-self-report measures of student engagement and accountability online were limited because the LMS inhibited us from taking digital measurements of individual students’ behavior (e.g., number of minutes spent watching the video, clicking behaviors). A second limitation includes the potential biases associated with researcher-coded qualitative responses. While we discussed all coding disputes between researchers, the qualitative nature of the questions makes it difficult to generalize findings beyond the context of this study.

Results

We present quantitative results, followed by qualitative, each organized by the four themes of interest identified post-hoc through the data analysis process. The four major themes include: student engagement, scaffolding learning, learning gains, and student accountability.

Quantitative Measures

Likert Questions

Responses to the scale items did not differ significantly as a function of the instructor or course semester, so quantitative analyses were collapsed across these variables. To examine students’ perceptions of the two digital storytelling formats, descriptive statistics were calculated. Table 1 displays the apriori survey items, organized around the four major themes identified at the conclusion of the study. Medians were used as the primary measure of central tendency because the survey used forced-choice methods, providing interval data. A more appropriate and conservative way to analyze such data is to assess medians and to use non-parametric tests. Overall, median ranks for both video formats were on the positive side of the scale (rated a “3” or “4”) for all nine items, indicating that students found both video formats “helpful” or “somewhat helpful”. Means are also reported, demonstrating the same trend.

Table 1 Frequency, median, mean, and standard deviation of participant ratings from survey responses
Full size table

Paired comparisons using Wilcoxon Signed Rank tests indicated that on all nine items, participants rated the interactive digital stories significantly more helpful than conventional digital stories (Table 2). Effect size correlations were calculated (Rosenthal 1994) and evaluated using Cohen’s (2013) criteria. Medium effect sizes (r = .3) were found for all three items that mapped onto student engagement. Similarly, two of the three items that mapped onto scaffolding learning also had medium effects, while the remaining item, “Helped you monitor your own understanding,” had a small effect (r = .1). The items that mapped onto learning gains and student accountability also represented small effects.

Table 2 Within subjects paired comparisons of perception of helpfulness for the interactive versus conventional digital storytelling formats
Full size table

Quiz Scores

Next, we explored the validity of the small effect on learning gains, which suggested that respondents believed they “retained more information” with interactive video. We compared mean quiz scores within subjects across the two quiz formats. Inclusion criteria were set such that participants were included if they completed quizzes for both types of digital storytelling formats across all 11 weeks of the semester that included video quizzes (n = 218). Note that in Semester A, only three of the 10 course sections used both video formats for all 11 weeks, because of instructor discretion (n = 51) while in Semester B, all students followed this format (n = 167). A single factor repeated measures ANOVA comparing mean quiz performance within individuals for the interactive versus conventional storytelling videos, across both semesters, indicated that responses did not differ significantly as a function of semester, so the data were collapsed across this variable. A paired samples test was then used to compare average quiz scores across the two video formats, within each of the participants. Results indicated that mean quiz scores were significantly higher for the interactive digital stories (M = 8.98, SD = 1.21) compared to conventional digital stories (M = 8.68, SD = 1.04), t (217) = 3.50, p < .001. This is a small effect (r = .133), according to Cohen’s (2013) criteria.

Qualitative Measures

Next, we evaluated participants’ responses to the two open-ended questions regarding the learning advantages and disadvantages of the interactive digital storytelling video.

Learning Advantages

Students were asked, “In your opinion, what are the learning advantages to the embedded quiz questions? What benefits (if any) did this format have on your understanding of the topic?” This question was answered by 182 participants, with 22 of the responses discarded for failure to address learning advantages of the embedded quizzes. Within the remaining 160 responses, we identified eight sub-themes that fit to the four major themes of interest (Table 3). Five of these sub-themes overlapped conceptually with the learning advantages in the Likert portion of the survey (comprising 78 % of the observed sub-themes). Of particular interest though, were the remaining sub-themes not expressed in the Likert items, as we did not predict these. Both the predicted and unpredicted advantages are discussed below, organized by the four major themes of interest and presented in descending order by percent frequency.

Table 3 Themes relating to participants’ perceived learning advantages for the interactive digital storytelling video
Full size table

Student Engagement

The affordance of student engagement was addressed by 61 % of student (n = 98). Consistent with the ideas presented in the Likert portion of the survey, in their open-ended feedback, students reflected the idea that the interactive video facilitated attention, focus, and engagement, because questions could “pop up” any time.

Scaffolding Learning

The next most prevalent theme in students’ responses was scaffolding learning (n = 88, 55 %). Two predicted and two unpredicted sub-themes fit into this category. Consistent with the concepts in the Likert portion, students commented that the interactive video “Facilitated re-watching and corrections” and “Facilitated monitoring learning”. The first unpredicted sub-theme was that interactive digital stories helped students identify what content was important as well as where that information was located in the video. Over one fifth of students (n = 33) explained, “It makes it a lot easier to track where the information is,” and that the quiz questions helped them “identify information that was most important from the video.” The next unpredicted sub-theme was that the interactive video was easier, more efficient, and/or scaffolded. Students expressed it was “Easier to answer questions one at a time instead of all at once,” that they “Didn’t have to write the quiz answers down and wait for them to be said in the videos,” and they could “Click the answer and move on.” The interactive features helped them, “Chunk the workload,” “Make the videos feel like they were in smaller, more easily understood segments,” and made students not feel “So overwhelmed.”

Student Accountability

The third most prevalent theme was promoting student accountability (n = 30, 19 %). Consistent with the idea in the Likert portion, students said interactive video ensured that they watch the entire video because a quiz question could “pop up” at any time.

Learning Gains

The theme of learning gains was also present in students’ responses (n = 26, 16 %), with both a predicted and unpredicted sub-theme. Consistent with the Likert items, students indicated that the interactive video improved short-term retention, as they were able to remember the information needed to answer the quiz, rather than forgetting it by the time the video ended. Furthermore, one in ten students (n = 18) reported they learned better from the interactive digital storytelling video, or that they saw enduring learning gains reflected by improved quiz scores on the interactive quizzes.

Learning Disadvantages

The second open-ended question asked, “In your opinion, what are the learning disadvantages to the embedded quiz questions? What drawbacks (if any) did this format have on your understanding of the topic?” We gathered responses from 184 participants, but 22 did not identify learning disadvantages of the embedded quizzes (e.g., general concerns about the digital stories and quiz questions that were not specific to the interactive video). In the remaining 162 responses, we identified five sub-themes that mapped onto the four themes of interest, and two new themes relating to other unpredicted challenges (Table 4).

Table 4 Themes relating to participants’ perceived learning disadvantages for the interactive digital storytelling video
Full size table

Notably, the highest frequency sub-theme identified was a belief that there were no disadvantages to the interactive video format (n = 59, 36 %). The next highest frequency sub-theme was student accountability (n = 29, 18 %), reflecting an idea contrary to the 19 % of participants who cited student accountability as an advantage in the previous question. Respondents citing accountability challenges indicated that it was possible to succeed on the interactive quizzes without having to watch all the assigned content in its entirety. One student explained, “Students can have video open and playing in one tab, not watch the video and just listen for when audio stops to go back a few seconds in the video to get quiz answers.” The remaining 34 % (n = 55) of participants reported unpredicted challenges related to technology glitches and usability, which are discussed below.

Technological Challenges

The first unexpected sub-theme related to technological malfunctions, including electronic point tabulation errors, answer submission failures, challenges with loading and opening the video, and general technological difficulties. One student described a reoccurring issue, that sometimes, “The video skipped a question and did not let me answer it, causing me to miss out on that point.” Another student explained that the videos were difficult to open, “Not everyone could load them on their first try… I had to borrow other family members’ computers to watch any of the video that had embedded quizzes.”

Usability Challenges

Another unexpected sub-theme related to usability issues associated with interactive video. Students addressed the inability to rewind or advance video, and change one’s quiz answers with ease, explaining, “It did not allow me to go back and review the material, so I could answer the question correctly.” Another student commented that, “If you wanted to change your answers, you had to go all the way back through the video after changing the answer.” In spite of these particular students’ beliefs, the reality was that all digital stories in the course supported complete student control in starting, stopping, fast forwarding, rewinding, and skipping across the video. Additionally, at the end of the interactive video students were given their full set of quiz question responses to change as necessary. Because these affordances were present in the video, but 10 % of respondents (n = 17) believed otherwise, these comments reflect the theme of usability issues.

Comparing Affordances and Challenges

In Table 5, we present a summary of the qualitative responses. Across the four themes of interest, the proportion of advantages cited overwhelmingly shadowed the frequency of disadvantages for the first three themes: student engagement, scaffolding learning, and learning gains, but the theme of student accountability had nearly equal representation of advantages and disadvantages.

Table 5 Comparison of participants’ perceived advantages and disadvantages by theme
Full size table

Discussion and Implications

Taking the qualitative and quantitative evidence together, we see students believed interactive digital storytelling facilitated engagement, scaffolded their learning, and supported learning gains. Students believed interactive video helped hold their attention, while presenting manageable chunks of content, and indicating what was important and where it could be found in the video. These findings are consistent with the experimental evidence suggesting interactive video supports learning gains in university students because they make learning active, engaging, and scaffolded (Cherrett et al. 2009; Delen et al. 2014; Merkt et al. 2011; Merkt and Schwan 2014; Zhang et al. 2006).

Second, by examining students’ quiz scores across the two formats, we found additional support for students’ belief that interactive video was associated short term learning gains. The presence of learning gains for the interactive video was consistent with existing experimental evidence (Cherrett et al. 2009; Delen et al. 2014; Merkt et al. 2011; Merkt and Schwan 2014; Zhang et al. 2006), but should be interpreted cautiously. Quiz scores may not have been a particularly informative measure since scores were high across the board, as the questions were intentionally designed to be learning checks. Indeed, the effect size of the paired samples test was small. These findings might indicate that the interactive video promoted short term learning gains within our sample, but we do not know if lasting learning gains were attained. Students’ expression that the interactive video supported long term learning gains cannot be validated with any data from our study. Future research investigating the long-term effects of interactive formatting might shed more light on this issue.

Finally, results were inconclusive regarding interactive digital storytelling’s role in promoting student accountability. One major reason we chose to use the interactive video was to increase student accountability for watching the online content, addressing a major issue facing university online and hybrid programs (O’Flaherty and Phillips 2015). Results indicated while some students believed the interactive video increased accountability (n = 30), others believed the opposite (n = 29). It seems the challenge of supporting students to be honest in completing work outside of FTF time remains an unresolved issue. As Hannafin (1985) explained, increasing interactivity is one possible solution, but as our findings suggest, this might not be true for all students. Indeed, a certain segment may be intent on successfully “gaming the system.”

Past research has touted the benefits of interactive video, but in this study, we examined its use within an authentic university hybrid course over the span of a 15-week semester and shed light on issues relating to its use. Students identified usability and technology issues as the biggest barriers. Consistent with Rafaeli’s (1988) concerns regarding design intentions versus individual user adoptions, we found that students’ interactions with the video were likely impacted by their beliefs about the interactive affordances (e.g., the subset of students who believed that they could not control video or change quiz answers, when in reality all students could perform these controls). One solution for both usability and technology issues is to structure student training on how to use the interactive features (Merkt and Schwan 2014) and solve basic hardware issues on their personal computers that may result in user errors. For example, in the SSFT course, we have added an in-class demonstration as well as a tutorial in the first week, to guide students through using the interactive video. Anecdotal evidence from using these support resources over the last year suggests that they have reduced the number of students experiencing these types of issues.

This study explored a first attempt at making digital storytelling video interactive. HapYak, as well as similar interactive video tools, Zaption, eduCanon, and EDpuzzle, offer other interactive features beyond multiple-choice quizzes, including embedded pop-up links, calls to action, open-ended quiz prompts, and branching/choose your own path through the content, among others. We find these particularly appealing, as they would bolster a more constructivist learning experience (Phillips 1995). Throughout our experiment, we learned that HapYak made it easy to create interactive digital stories with limited time, money, and personnel. However, there were some challenges that included LMS integration, technical glitches, and sufficient student and instructor support. Our biggest concerns were integration across various learning management platforms, and with Zaption, potential legal issues of content ownership and storage. Developers must work through these issues before universities can adopt such tools at a large scale.

In summary, in the context of a semester-long hybrid course on sustainability science for preservice teachers, the vast majority of students viewed interactive digital stories as a meaningful way to engage and scaffold their learning. Analysis of quiz scores indicated that students scored significantly higher for the interactive versus conventional video format, although the effect size was small. These results extend evidence of the success of interactive video to an authentic university class setting over a complete semester. Nevertheless, unresolved questions remain regarding if and how interactive video might address the issues of pedagogical integrity and support meaningful and/or lasting learning gains. Despite these issues, within the SSFT course, interactive digital storytelling provided a powerful alternative to conventional text and lecture methods, which are not without their own challenges. Within the growing body of hybrid university courses, interactive digital storytelling represents a useful and relevant way to engage millennial learners in an online learning environment.