Introduction

Research on game-based environments has predominantly focused on investigating the effect of gameplay on learning without a systematic examination of the design features and the development processes (Clark et al., 2016; Ke, 2016). To create a robust scientific foundation for designing educational games, scholars must report comprehensive descriptions of their development experiences by elaborating on the decisions and strategies grounded on theoretical foundations, along with recommendations and lessons learned (Ke, 2016). Additionally, research-based recommendations on the application of multimedia learning principles are scarce in the literature (Churchill, 2013). To shed light on this matter, we describe the design and evaluation processes of 18 learning support videos for an educational physics game and report how multimedia learning (Mayer, 2017) and instructional design (Merrill, 2002) principles facilitated the development of the videos. We conclude with the results of the effectiveness of our final videos and recommendations for future research and practice.

In-game learning supports

In-game learning supports can aid learners' cognition during gameplay, helping them focus on important information, figure out what to do next, and generally engage in more efficient learning (Wouters & van Oostendorp, 2013). On the other hand, poorly designed learning supports can disrupt gameplay, demand more cognitive effort to connect content knowledge to game tasks, and may not promote learning (Schrader & Bastiaens, 2012). Thus, mixed results concerning in-game learning supports are found in the literature. For example, in a math game with learning support videos, Delacruz (2010) found that learners who watched the support videos outperformed the control group in the far-transfer test controlling for pretest scores. Wouters and van Oostendorp (2013) conducted a meta-analysis and found a moderate effect of learning supports that combine visual and auditory forms (e.g., videos) on learning. Conversely, Van Eck and Dempsey (2002) reported no effect of learning support videos in a geometry game, showing no significant correlation between transfer scores and support usage frequency.

These mixed results regarding the effectiveness of learning supports might be due to the varied designs of each learning support and the type of content involved (Clark et al., 2016). Clark et al. (2016) point out that although games as a medium provide affordances, it is the design of the medium that will determine its effect on learning. Additionally, the authors argue that we should shift from questions such as "Can games support learning?" or "Are games better with or without learning supports?" to explore how design decisions grounded on theoretical foundations influence learning outcomes concerning the wide diversity of learners. Thus, through experimentation and discourse, researchers and practitioners can develop a strong foundation for designing effective in-game learning supports, anticipating errors, and making efficient design decisions (Richey & Klein, 2007). To contribute to this foundation, we examined the optimal design of in-game learning support videos for learning conceptual physics, resulting in recommendations and suggestions for future research and practice.

Multimedia learning principles

Over the past two decades, Mayer and colleagues have compiled a set of principles for designing multimedia instructional materials, defined as a presentation composed of words (e.g., narration) and pictures (e.g., animations) developed to foster meaningful learning. According to the Cognitive Theory of Multimedia Learning (Mayer, 2017) and Cognitive Load Theory (Sweller et al., 2011), people have two separate information processing channels (i.e., auditory and visual) and working memory that is resource-limited. Due to this limited capacity, multimedia instructional materials must present the content without overloading the visual and auditory channels in working memory to facilitate cognitive processing (Mayer, 2017; Schwan et al., 2018). Researchers (e.g., Mayer, 2017; Sweller, 2020) have thus explored the use of multimedia learning principles to achieve this balance by addressing three fundamental objectives: (a) reduce extraneous load, which is unnecessary cognitive processing generated from poorly designed instruction; (b) manage essential cognitive processing, which refers to constructing mental representations of the material in the working memory; and (c) foster generative cognitive processing, relative to deep learning and making sense of the materials, enabling both retention and transfer.

Within each objective, principles are identified that address the objective. For example, the spatial contiguity principle, intended to reduce extraneous load, states that people learn better when corresponding words and graphics are located near each other rather than far from each other (Johnson & Mayer, 2012; Mayer, 2017). Using an eye-tracking method, Makransky et al. (2019) found that learners engaged in more appropriate cognitive processing in lessons with the spatial contiguity principle than without the principle, as learners spent more time looking at the text and less time looking at irrelevant parts of the illustration. Further, the modality principle, related to the second objective, states that people learn better from graphics with narration than on-screen text (Mayer, 2017). For instance, Schwan et al. (2018) found that participants in an art exhibition are more likely to remember the paintings when the exhibition was designed using narration via an audio guide rather than extended written information. The modality principle helps learners process the content using both visual and auditory channels, off-loading parts of the cognitive processing from the visual to the auditory channel (Mayer, 2017; Moreno & Mayer, 2002; Sweller et al., 2011). For the third objective, the multimedia principle is an example of principles intended to foster generative cognitive processing. The multimedia principle states that people learn better from words and graphics than words alone, helping learners connect and make sense of verbal and visual mental representations (Mayer, 2017). Studies showed that learners in multiple-representation conditions (i.e., composed of words and graphics) outperformed those who studied lessons with words alone on retention (e.g., Moreno & Mayer, 2002), transfer (e.g., Moreno & Ortegano-Layne, 2008), and recall tasks (Glaser & Schwan, 2015).

When designing multimedia instructional materials, addressing more than one objective through multiple principles can enhance cognitive processing and associated learning outcomes. Hence, this study combined multiple principles in developing in-game learning support videos. Table 1 shows our focal nine principles.

Table 1 Focal nine multimedia learning principles in this study (
Full size table

First principles of instruction (FPI)

Merrill (2002) systematically reviewed various instructional system models, design theories, and research and practice related to learning and instruction to identify underlying mutual principles. To be selected, the principles had to satisfy the inclusion criteria. They needed to: promote efficient, effective, and engaging learning; be applicable in any delivery system; and be design-oriented (i.e., intended to guide the development of learning environments and products rather than explain how learners gain knowledge or skills from these environments or products). The results from his extensive review identified five principles, known as the First Principles of Instruction (see Table 2). Researchers have subsequently examined various learning environments and products designed with the First Principles of Instruction (FPI) and/or multimedia learning principles (Chiu & Churchill, 2015; Lo et al., 2018), discussed next.

Table 2 First Principles of Instruction (
Full size table

Applying research-based principles

Many educational studies often focus on the effects of research-based principles without a rigorous examination of the design and development processes. However, these examinations serve as an important link between theory and practice by offering a more comprehensive knowledge of the field and precedents to make predictions (Richey & Klein, 2007). Chiu and Churchill (2015) applied several multimedia principles in developing mathematics lessons. They recommended objective guidelines based on their results, such as using different colors for each lesson section and placing graphs next to equations. In a later study for algebra learning, based on the data from interviews with students, Chiu and Churchill (2016) recommended using color matching to signal related pieces of information and adding graphics (e.g., dots) to indicate important parts of a graph.

Likewise, research-based recommendations for applying the FPI were also examined. Lo and Hew (2017) used the FPI and multimedia learning principles in designing instructional videos for mathematical learning. They recommended limiting the videos' duration to six minutes and presenting a brief review of key concepts. Further, Gardner et al. (2020) applied the FPI in designing digital materials and recommended including realistic examples from various contexts for novice students and creating multiple practice opportunities. Also, Lo et al. (2018) and Klein and Mendenhall (2018) suggested considering time constraints for developing instructional videos. Moreover, Tu and Snyder (2017) and Lo et al. (2018) warned that using the FPI to create well-designed materials does not guarantee learning outcomes if students lack motivation. Therefore, motivational strategies should also be considered in the design process.

Practical recommendations from research are essential to blur the line between practitioners and researchers. However, recommendations on applying research-based principles for designing effective in-game learning support videos are scarce in the literature. Thus, the objectives of this study were to (a) examine the optimal design of applying the First Principles of Instruction and multimedia learning principles to develop in-game learning support videos for learning conceptual physics; (b) evaluate the effectiveness of the videos on learning content knowledge, solving game levels, and game enjoyment; and (c) propose recommendations for future practice and research.

Method

Design

We used a case study method to explore the in-depth application of First Principles of Instruction and multimedia learning principles on our design decisions and evaluate the final product. A case study is one of the various methods of design and development research, which aims to construct knowledge based on scientific evidence obtained from practical experiences and includes a systematic analysis of the design, development, and evaluation processes (Richey & Klein, 2007).

Participants

This study comprised two stages: design and evaluation. In the design stage, the research team included: (a) two faculty members in Educational Technology responsible for creating the instructional strategies for the videos and, along with one faculty in Measurement and Statistics, revising all videos biweekly to guarantee they followed the design parameters; (b) two Subject Matter Experts (SME) in physics responsible for ensuring the content was clear, concise, and accurate; and (c) five graduate research assistants responsible for editing the videos. Two graduate assistants reported having basic to intermediate video-editing skills, and the other three had no prior experience. The former provided training to the latter, targeting skills such as overlaying text and working with keyframes. After the training, all five graduate assistants independently produced in-game learning support videos, which we call physics videos in the current context.

In the evaluation stage, we included data from 14 middle school students from a charter school and 263 high school students from a large K-12 school, both selected through convenience sampling in the southeastern United States. All students submitted their signed parental consent and assent forms.

Procedure

Figure 1 summarizes our research procedure. In the design stage, we applied the First Principles of Instruction (Merrill, 2002) to create instructional strategies such as presenting demonstrations of failed and successful attempts on game levels. Next, we used various multimedia learning principles (Mayer, 2017) to make design decisions (e.g., removing extraneous graphics) to help learners engage in cognitive processing. In the evaluation stage, we conducted: (a) alpha testing with internal tests to iteratively revise the videos, (b) beta testing to test the initial seven videos with 14 students and gather feedback on their perceptions, and (c) user-acceptance testing to examine the effectiveness of the final videos on learning content knowledge, solving game levels, and game enjoyment with 263 students. We spent two months developing the initial seven videos, and, after the beta testing, we spent six months revising and developing all 18 videos. To obtain in-depth information on how designers used the FPI and multimedia learning principles in designing the physics videos, we analyzed all notes documented between 2017 and 2019, including the usability reports, and reflected on our experiences to produce recommendations for researchers and practitioners.

Fig. 1
figure 1

Research procedure in this study

Full size image

Data Source

We employed qualitative techniques to collect data through two sources: (a) content analysis of detailed notes from the research team meetings and (b) observations and reports from usability testing. We also included quantitative techniques (i.e., satisfaction survey, physics understanding test, and log files) to gather feedback on students’ perceptions and examine the effectiveness of the physics videos. The physics understanding tests included illustrative multiple-choice items split between two equivalent forms for a pretest and posttest. The satisfaction surveys included 5-point Likert scale items about game satisfaction ranging from strongly disagree to strongly agree (e.g., "I enjoyed the game very much"). The log files were recorded while students played the game, and we parsed the log files and computed variables such as the frequency of accessing the learning supports and the levels completed for each student.

Educational game

Physics Playground is a 2-dimensional computer-based game designed to help students learn conceptual physics such as Newton's laws of force and motion, torque, and energy (Shute, Almond, et al., 2019; Shute, Ke, et al., 2019). The game consists of two types of game levels: sketching and manipulation. In both level types, the goal is to move a green ball to hit a red balloon. To solve sketching levels, students draw simple machines (i.e., ramps, levers, pendulums, and/or springboards) directly on the computer screen that interact with the game environment according to Newtonian mechanics (Fig. 2). To solve manipulation levels, students adjust different sliders to change physics parameters (i.e., gravity, air resistance, mass, and bounciness of the ball) and interact with external forces such as puffers and blowers (Fig. 3).

Fig. 2
figure 2

Sketching level – to solve the level, learners can draw a springboard. See https://youtu.be/5mJGI7ty2Wk

Full size image
Fig. 3
figure 3

Manipulation level – to solve the level, learners have to manipulate the air resistance slider. See https://youtu.be/KQ9ACpqLxCU

Full size image

Results

First, we report the alpha testing results regarding the optimal design of applying the FPI and multimedia learning principles in developing in-game learning support videos. Next, we present the beta and user-acceptance testing results concerning students' perceptions and the effects of the videos on learning content knowledge, solving game levels, and game enjoyment. We conclude with a discussion of recommendations for future practices and research.

Alpha testing

Alpha testing includes internal tests with content experts to identify all possible issues before releasing a product (Mohd & Shahbodin, 2015). Over three years, we used an iterative process to create and validate several learning supports in Physics Playground. Results from our first two studies (Shute, Ke, et al., 2019) and researchers’ observations revealed the need for a new type of learning support to more closely connect how students solve a level to the physics involved in the solution. Thus, we decided to create the physics videos to connect each intersection of solution (e.g., ramp) to the relevant competency (e.g., Newton's 1st Law) occurring in game levels (see an example: https://youtu.be/cewsive2D0U).

First, the physics experts examined all 81 game levels and identified 18 appropriate intersections for the physics videos. Afterward, we reviewed the FPI (Table 2) to define instructional strategies for the videos. For example, based on the activation principle, we opted to use the tutorial levels to capture the gameplay footage, as seeing these levels in the physics videos could activate students' prior knowledge about the referenced game mechanics. This prior knowledge can act as the foundation for building the formal physics knowledge students are acquiring through gameplay, highlighted in the physics videos. Moreover, instead of explaining the physics concepts in a direct way (e.g., presenting the definition of a concept), we demonstrated the physics concepts by showing a failed attempt (non-example) followed by a successful attempt (example). The successful attempt models the correct action or behavior (Merrill, 2002), an important aspect of the demonstration principle. Thus, each physics video followed the same format: (a) introduction of the physics competency (e.g., "Here you are going to see how to transfer energy to the ball using a pendulum"); (b) definition of terms (e.g., "Kinetic energy is the energy of motion…"); (c) failed attempt to solve the level (e.g., "The pendulum does not have enough angular height…"); and (d) correct action (i.e., changing the height of the pendulum) to show a successful attempt to solve the level. Another strategy, based on the application principle, was to embed the relevant physics video in each corresponding level, so students have the opportunity to apply what they learned immediately after watching the video. The relevancy of the physics videos to their associated game levels enables the immediate and purposeful application of the new knowledge.

After planning the strategies, the development of the physics videos followed five stages:

Scripting

In previous studies (Shute, Ke, et al., 2019), students had access to a set of Hewitt videos that consist of animations explaining general physics competencies such as Newton’s Laws, created by Paul Hewitt. Based on researchers’ observations, most students did not watch the whole Hewitt video. When asked why they didn't finish, students mentioned that the videos were too long. One student was even surprised to learn that the Hewitt video was only around 2 min long. Thus, for the physics videos, we limited the length of each video to one minute. With that in mind, the physics experts created a script for each physics video. They included concise narration for the competency definition, the failed and successful attempt, and direction for the game footage needed to illustrate the narration. In addition, based on the personalization principle, the narration addressed the player using "you" and "we," for example, when introducing the physics concept (e.g., "Here you are going to see how mass affects the equilibrium of a lever") or when providing explanations (e.g., "You need to draw another pendulum with more mass").

Storyboarding

The graduate assistants created storyboards for each video based on the scripts. They first created slides presenting the game footage for each segment of the narration with the proposed text or graphics overlays. Each storyboard had to be approved by the faculty members and physics experts before starting the video editing. Since video editing is the longest step in developing the videos, revising and approving the storyboards were essential to optimize the process and avoid significant revisions in editing the videos.

Audio recording

Once the storyboard was approved, we recorded the narration. Our decision to use narration rather than on-screen text was based on the modality principle – people learn better from graphics with narration than graphics with on-screen text (Mayer, 2017). Also, extensive research on the modality principle contributed to uncovering boundary conditions (i.e., specific conditions under which the principle is effective) (Mayer, 2017). For example, we opted to use narration to deliver the verbal information along with on-screen text only when introducing/defining physics concepts (e.g., kinetic energy), following studies that suggested using on-screen text to present unfamiliar or technical words (e.g., Harskamp et al. 2007).

Although multimedia principles can serve as heuristic guidelines to make reasonably rapid theoretically-driven design decisions, the principles are not valid for all the wide variety of settings, learners, and contents. Thus, designers must consult the validated boundary conditions to identify when to use and when to violate the principles. For example, one team member recorded all narrations to guarantee consistency and alignment with the voice principle – people learn better from a friendly human voice rather than a machine-like voice. However, examining the content analysis, we noted the absence of discussion on intonation, rhythm, pace, and pitch due to the lack of boundary conditions regarding these features for the voice principle. Although we used an instructive tone and rhythm of speech to offer verbal cues, the decision was not methodically discussed. We concluded the decision was based on the previous instructional experience of the team member who recorded the audios.

Video editing

We synchronized the narration with the gameplay footage and on-screen text following the temporal contiguity principle. Instead of displaying the complete formula "momentum = time × velocity" after the narration, we displayed each word as it was spoken. When the narration is presented before words or graphics, learners must hold the narration in their working memory until the words or graphics are presented, which reduces the cognitive capacity to make sense of both information sources (Mayer, 2017).

We also limited the amount of on-screen text to align with the redundancy principle – people learn better from a combination of graphics and narration than from a combination of graphics, narration, and on-screen text (Mayer, 2017). We used narration alone rather than narration and on-screen text, except when presenting unfamiliar words (i.e., physics concepts). For example, when introducing "Kinetic Energy," learners would hear and see the physics term simultaneously. This decision aligns with studies that found redundancy can promote learning when on-screen text is reduced to a few words (e.g., Harskamp et al. 2007). Hence, we only used on-screen text to present unfamiliar terms that would otherwise not be fully processed by the auditory channel alone (Fig. 4).

Fig. 4
figure 4

On-screen text was limited to physics concepts and placed near the related part of the graphic

Full size image

Since our game is responsive (i.e., the layout automatically adjusts to different screen sizes), we noticed the need to record gameplay footage using the same type of device and web browser to ensure consistency in footage aspect ratio and resolution. We used the game's tutorial levels to capture gameplay footage for the videos. Tutorial levels contain only essential graphic elements, as opposed to other levels with elaborative drawings. Thus, we employed the coherence principle by omitting extraneous graphics to help learners focus on the physics explanations. We applied the spatial contiguity principle demonstrating the change in physics variables (e.g., kinetic energy) during gameplay. We first prototyped animations of meters that would fill and empty according to the ball's movements. However, we noticed a potential split-attention effect, meaning that learners would be forced to split their attention between the meters and the physics variables and mentally integrate the two sources of information (Chandler & Sweller, 1992; Johnson & Mayer, 2012). Thus, to present how the physics variables change according to the ball's movements, we animated the on-screen text to move with the ball, and the font size would increase or decrease to represent the change in magnitude (Fig. 5). We also applied the visual design principle of similarity (i.e., elements with common characteristics are perceived as related) to enhance the connection between on-screen text and game elements. For example, the color of the text would be green when related to the green ball (Lauer & Pentak, 2011) (Fig. 6).

Fig. 5
figure 5

Sequential images showing the application of the spatial contiguity principle. Note. GPE = Gravitational Potential Energy, KE = Kinetic Energy

Full size image
Fig. 6
figure 6

Application of the similarity principle

Full size image

Lastly, we noticed the need to use the signaling principle to move learners' attention from the ball to the mouse movements interacting with the blower. This design decision was necessary because, otherwise, learners would pay attention to the ball, the protagonist in our game, while the physics explanation focused on manipulating the blower. We created a hue contrast by placing a semi-transparent black layer on the screen, leaving a spotlight where students should focus (Fig. 7).

Fig. 7
figure 7

Spotlight to signal where students should focus

Full size image

Revisions

The research team iteratively revised each new video. As we developed more videos, we gained more insight for improvement and applied these insights to previously developed videos. Hence, all videos went through several rounds of revisions. Additionally, although we discussed and documented the design parameters for editing the videos, designers used different approaches to follow the parameters. For example, two designers used bitmap images for on-screen text, while others used the actual font, causing the text resolution to look slightly different from one video to another. To avoid further redesigns, we recommend using a template file from the outset to serve as a demonstration of how to perform tasks instead of written parameters that merely say what to do. After this design cycle, the videos were ready for beta and user-acceptance testing, discussed next.

Beta testing

Beta testing implies using the complete product by a few representative users in a real environment to gather feedback on product quality (Mohd & Shahbodin, 2015). We conducted the testing with 14 middle school students (6 seventh graders, 8 eighth graders) in a charter school in the southeastern United States (Shute, Smith, et al., 2020). Participants were recruited through a convenience sample and played the same game with 30 sketching levels and seven physics videos for 75 min. Students had access to the videos at any time during gameplay, and, at the end of some levels, a popup window would appear to present a physics video. All students completed a satisfaction survey and were compensated with a $10 gift card upon completing the study. A total of 5 researchers observed the students and took various notes on students' reactions, commentaries, and gameplay.

Despite the limitations (i.e., small sample size and short gameplay time), we obtained useful insights to improve the physics videos. We also looked at the satisfaction survey to see how students felt about physics videos (Table 3). In general, students found the videos satisfying and useful (M = 3.99, SD = 0.51) and believed the videos helped them learn physics (M = 3.79, SD = 1.19). Table 4 shows selected commentaries from students. One student indicated liking the videos for not showing the exact solution, and another student pointed out the videos helped solve multiple levels. Four students mentioned that the videos were not related to the levels they just played, and three students reported preferring to watch the videos at the beginning of the level. Based on the feedback, the physics experts revisited each game level’s connection to the physics competencies in the game to ensure all levels had the appropriate physics video embedded. For the interaction, we removed the popup window presenting the videos and preserved free access to the videos. Additionally, researchers noted that most students watched the entire video when accessing the physics videos. Based on these results, we continued developing the remaining physics videos following the same process.

Table 3 Learning support satisfaction scale (n = 14)
Full size table
Table 4 Examples of students' commentaries and design modifications
Full size table

User-acceptance testing

User-acceptance testing is performed by the end-users, and it is intended to verify whether the desired goals were met before launching the product into the audience's life (Mohd & Shahbodin, 2015). We conducted the user-acceptance testing with 263 high school students from a large K-12 school in the southeastern United States (Shute, Rahimi, et al., 2020; Shute, Smith, et al., 2020). Participants played the game with 81 game levels (sketching and manipulation) and all seven supports (Table 5), including the 18 physics videos, across six days in 50-min sessions per day. They also completed a pretest (α = 0.77), posttest (α = 0.82), and satisfaction survey (α = 0.67) and received a $30 gift card.

Table 5 Description of the seven learning supports in the game
Full size table

We computed regression analyses predicting posttest scores with each learning support frequency as the predictor, controlling for pretest. Results revealed that, among all supports, the physics videos were the only support that significantly predicted posttest scores (F(2, 198) = 97.46; p < 0.001, β = 0.11; t = 2.10, p = 0.04) and game level completion (F(2, 198) = 40.63; p < 0.001, β = 0.32; t = 5.14, p < 0.001) (Table 6 and 7). In addition, we found no significant difference in game enjoyment between students who did not watch, watched a few, or more than five physics videos (F(2, 192) = 1.89, p = 0.15, partial η2 = 0.02). The satisfaction survey results were consistent with beta-testing as students found the videos satisfying and useful (M = 3.58, SD = 0.72) and believed the videos helped them learn physics (M = 3.56, SD = 1.09). Based on the log files, we found that students watched the same physics videos multiple times, showing that they could perceive the value of watching physics videos. These findings suggest that the physics videos were effective in promoting learning and game performance without disrupting gameplay or reducing enjoyment.

Table 6 Coefficients table of regression analysis with posttest score as the dependent variable
Full size table
Table 7 Coefficients table of regression analysis with game levels completion as the dependent variable
Full size table

Discussion

We examined the use of research-based principles in developing learning support videos and evaluated the effectiveness of these videos in promoting learning and game performance without disrupting gameplay. The results of our iterative design process suggest the following recommendations for future research and practice.

Recommendations for designing instructional strategies

In-game learning support videos should present the connection between targeted content knowledge and game mechanics. The physics videos were the only support designed to target both physics concepts and gameplay. Accordingly, physics videos were the only type of support that significantly predicted posttest scores and game level completion, controlling for pretest. This finding is consistent with Delacruz (2010), who created tutorial videos targeting math concepts within the game mechanics and found a positive effect on far-transfer test scores. This finding also supports Ke’s (2016) arguments for blending learning and gameplay intrinsically when designing games and learning supports.

In line with Gardner et al. (2020), who suggested creating multiple practice opportunities in digital settings, we recommend integrating the relevant video in each game level to give students the opportunity to apply what they learned right after watching the video. However, the most beneficial timing to present the videos is still unclear (e.g., before or after playing the level, or when stuck in a level). Future research is needed to identify the appropriate time to present the videos. Researchers may also consider exploring an adaptive delivery of learning supports, such as predicting when and how students need to watch the videos according to their gameplay progress.

Further, in-game learning support videos should be limited to one minute to increase the chances of students watching the entire video and minimize gameplay interruption. Based on reports from previous studies, students did not watch the whole Hewitt videos because they were too long (about 2 min). Although the Hewitt videos' content is different from the physics videos, we observed that students finished watching the physics videos limited to one minute in the beta testing. In addition, during the user acceptance testing, we found students watched the same physics videos multiple times, suggesting video length was not an issue. This recommendation supports Nielsen's (2014) findings that a 2-min demonstration video can be too long and does not add substantial value over a 1-min video.

We also recommend designing learning support videos with the same look and feel as the game to help activate students’ prior knowledge. For example, use tutorial levels as the setting to activate prior knowledge about the referenced game mechanics. Additionally, like Lo and Hew (2017), we suggest adding a brief review of the targeted concepts to activate prior knowledge. Next, we suggest adding demonstrations of a non-example and example of how game or content variables impact the solution. Showing a common failed attempt followed by a successful attempt illuminates what factors lead to failure and what factors lead to success, a possible reason why the physics videos were effective for solving levels.

Recommendations for developing in-game learning support videos

We recommend placing on-screen text (e.g., GPE) next to graphics (e.g., ball) and maintain their proximity throughout the animation (i.e., animate the on-screen text to move with the ball). This recommendation corresponds with Chiu and Churchill's (2015) suggestion to place graphs next to equations. However, in contrast to their materials, the graphics in the physics videos were in constant motion. Thus, for animations, designers can set various keyframes for time and position to synchronize the on-screen text with graphics, following the spatial contiguity principle. Additionally, when moving the on-screen text, we recommend changing the font size to represent the change in the variables' magnitude. Scaling font to illustrate variations relates to data visualization techniques (e.g., word cloud), and it is widely applied in real-world situations to facilitate semantic understanding (Yang et al., 2020). Future research may look at additional data visualization techniques such as variation in color tones and weight to demonstrate how physics variables change for students.

Further, we recommend using a visual cue, such as a spotlight (i.e., graying out unimportant parts at a particular moment) to signal where students should focus during a video, especially when attention to a specific detail is the critical part of the animation. In alpha testing, we noted that even we missed part of the animation without highlighting and directing our attention. This suggestion is consistent with Chiu and Churchill's (2016) recommendation of adding graphics to indicate key parts of a graph and supports Alpizar et al.'s (2020) results in a meta-analysis of signaling principle showing a moderate effect (d = 0.31) of using color contrast to highlight information.

To optimize the development process and reduce redesigns, we recommend creating and validating a storyboard before editing the videos. Revising the content during the storyboarding phase is faster than altering content in video editing, which could demand new audio recordings and gameplay footage. We also recommend using a file template in addition to documentation of design decisions (i.e., design parameters) to ensure consistency across videos edited by different designers and minimize redesigns (Farrell, 2015). A template serves as a demonstration of design methods – an approach related to the demonstration principle (Merrill, 2002).

Consulting boundary conditions for each multimedia principle is a key component of many design decisions since the principles are not valid for all types of settings and learners. These conditions helped identify when to use and when to violate the principles. However, our reports noted a lack of boundary conditions for the voice principle, resulting in scarce discussions about additional features such as intonation and pace. Thus, future research might consider exploring the boundary conditions regarding the voice principle to inform designers on decisions regarding intonation, rhythm, pace, and pitch. For example, Davis et al. (2019) found that other factors such as prosodic elements (i.e., rhythm and sound) might have a greater effect on the voice principle rather than just categorizing into human and machine voices. Also, Craig and Schoeder (2017) found no significant difference when the machine-voice is generated from modern text-to-speech engines that resemble human voices.

Finally, the log data indicated that students accessed the physics videos multiple times, suggesting that students could perceive the value of watching physics videos. According to Ryan and Deci (2000), this perceived value is known as identified regulation, a level of extrinsic motivation. Identified regulation is different from intrinsic motivation since the latter refers to performing a task because it is enjoyable, while identified regulation refers to doing the task because it will be beneficial. In other words, watching the physics videos enabled students to exert effort toward solving levels. These findings support Moreno and Mayer's (2007) discussion that learning is also mediated by motivational factors that increase or decrease cognitive engagement. Also, the repeated access of physics videos backs the discussion on maintained situational interest (i.e., when interest is held, and people start to connect with the content). Aligned with Dousay's (2016) findings on the impact of modality and redundancy on maintained situational interest, the right balance of animations, narration, and on-screen text in the videos might have helped students maintain situational interest, helping them manage intrinsic processing and engage with the content. Moreover, we found no difference in game enjoyment between students who watched a few or many videos, suggesting that the physics videos did not disrupt gameplay and enjoyment. Future research may further examine the effects of the various design principles on motivation and situational interest concerning learners' prior knowledge and other characteristics.

Limitations and conclusion

Research-based recommendations for designing game features based on comprehensive examinations of design experiences and grounded on theoretical foundations are needed (e.g., Clark et al., 2016; Ke, 2016; Moreno & Mayer, 2007). To address this need, the current study reported a detailed description of our design and evaluation processes for developing in-game learning support videos for physics learning. Our examination resulted in several recommendations for future practice and research. However, the study has limitations to consider when applying our recommendations, such as a small sample size and short gameplay time in beta testing and the lack of a control group to confirm the effects of each design element on learning (e.g., show the video with the same look as the game environment) in the user-acceptance testing. In summary, our recommendations include (a) showing the connection between how students solve a level to the learning content involved in the solutions, (b) demonstrating a failed and successful attempt, (c) intrinsically integrating support videos in the game environment, (d) delivering the relevant video in its connected level to relate to students immediate challenge, and (e) consulting boundary conditions to apply principles aimed to reduce extraneous load, manage cognitive processing, engage in generative cognitive processing, and maintain situational interest. Such careful designing and developing of learning support in educational games can help overcome the challenge many game-based researchers have been facing—maximizing learning without sacrificing the fun (Shute, Rahimi, et al., 2020; Shute, Smith, et al., 2020).