Keywords

An educational setting of interactive science multimedia involves at least two agents: the student and the multimedia. Both agents interact by giving and receiving information. Multimedia instructional designers aim to enhance these exchanges of information (see Domagk et al. 2010) when producing the myriad of interactive educational products available. Moreno and Mayer (2007) proposed a classification of this diverse offer of multimedia products, based on three levels of increasing interactivity:

  • Level 1: Low interactive tools entail students dialoguing with the multimedia. For example, learners can solve a problem presented on-screen. This level of dialoguing is generally included in computer-based instruction, digital libraries, and similar multimedia presentations.

  • Level 2: Medium interactive tools allow learners controlling the multimedia. For example, students can pause and rewind a presentation. This level is generally included in animations and videos.

  • Level 3: High interactive tools allow students manipulating the multimedia. For example, learners can drag objects around the screen. Manipulating is generally included in simulations and videogames.

In short, simulations and videogames allow the greatest interactive capabilities between the students and the multimedia. As the bottom levels contain the top ones, simulations and videogames allow students all three degrees of action: dialoguing, controlling, and manipulating. Given this high potential for interactivity, simulations and videogames are the focus of the current chapter.

Although there are two agents involved in an interaction event, educational research on interactive science multimedia must focus on the first agent, the student. As warned by Mayer (2014), multimedia learning research should primarily investigate the students’ mental processes, rather than the multimedia’s capabilities. This approach can help ensure that multimedia design supports mental processes necessary for meaningful learning.

However, there are many mental processes in which students engage during interactive science multimedia learning. As described in the integrated model of multimedia interactivity (INTERACT; see Domagk et al. 2010), the students are involved in motivational, behavioral, and cognitive processes. In this chapter, we will focus on cognitive processes and specifically the role of students’ visuospatial processing. We begin by describing different research perspectives that can help foster effective student–multimedia interactions.

6.1 Research Perspectives on Fostering Effective Interactivity

In all interactive multimedia learning situations, including those about health and natural science topics in university education, the student and multimedia are agents that exchange information. In a dialogue of giving and receiving information, the student engages in an action, which is responded by the multimedia as feedback, so the learner can generate a more appropriate action, receive feedback again, and so forth (see Domagk et al. 2010). Different research areas have investigated analogous dialogues in educational settings. We will describe them here as engagement by the student and feedback by the multimedia (see Fig. 6.1). As we note next, findings from research on engagement and feedback have provided guidelines to foster more effective learning with interactive science multimedia.

Fig. 6.1
figure 1

Agents, actions, and degrees in an interactive multimedia learning event

Full size image

6.1.1 Student Engagement

Chi and Wylie (2014) described the ICAP framework of cognitive engagement. This framework can be linked to other educational approaches, such as generative learning (see Fiorella and Mayer 2015, 2016) or embodied cognition (see Castro-Alonso et al. this volume-c, Chap. 7; see also Castro-Alonso et al. 2015). These research perspectives share that higher cognitive engagement can boost learning. As framed by the ICAP model (Chi and Wylie 2014) and discussed by Mayer (2014), it is important to distinguish between cognitive and behavioral engagement (e.g., hands-on activities). Although these types of engagement can be related, cognitive engagement is the primary driver of understanding, whereas behavioral engagement per se does not necessarily enhance learning. Thus, in general, higher cognitive engagement, independent of behavioral activities, will lead to higher quality learning outcomes.

The ICAP framework distinguishes among four degrees of cognitive engagement (see Fig. 6.1): interactive, constructive, active, and passive (ICAP). The framework predicts that learning activities fostering constructive and interactive cognitive engagement will be more successful than activities that only foster passive and active engagement. For example, the framework promotes science multimedia that allow both constructive engagement, in which students develop new ideas, and interactive engagement, in which students test those ideas and receive feedback from the multimedia. Note that these two top levels of engagement in the ICAP framework correspond to the highest level (Level 3, see above) of the model by Moreno and Mayer (2007). As such, constructive and interactive cognitive engagement is characteristic of learning through simulations and videogames.

For interactive multimedia, the evidence with university students shows that creating higher levels of engagement can produce better learning outcomes (e.g., Wiley 2019). Much of this research supports active over passive engagement. For example, Erhel and Jamet (2006) investigated 72 psychology undergraduates (85% females) learning health science topics from multimedia modules presented in different formats. Results showed that the most effective design was the one allowing students to click parts of the images to reveal corresponding blocks of texts (active), as compared to conditions in which clicking was not allowed (passive). In a study by Evans and Gibbons (2007), 33 business undergraduates (33% females) studied multimedia depicting the mechanisms of a bicycle pump. Students were randomly assigned to learn from a lesson without interactive features or to learn from a lesson that allowed the engagement activities of: (a) controlling the pace of the presentation, (b) testing questions, and (c) triggering a simulation of the pump. Results indicated that the interactive group outperformed the non-interactive group on a subsequent transfer test.

Schwartz and Plass (2014) compared passive, active, and constructive levels of engagement. They presented 112 university students with a multimedia lesson that narrated action phrases (e.g., “paint the fence”). Students were randomly assigned to one of three engagement conditions: (a) look (passive), in which images for the action (e.g., paintbrush and fence) were shown as static images; (b) click (active), in which participants pointed and clicked the images, and these actions resulted in the images becoming animated for the action; and (c) drag (constructive), in which participants dragged the images in order to generate the action by themselves. Recall and recognition tests for the action phrases showed that the best performance was for the drag condition, followed by click, and then look. In other words, consistent with the ICAP framework, the possibility of generating dragging movements on the screen (constructive) was better for memory tests than watching the movements already performed after clicking (active), which was also better than solely looking at the screen (passive).

Nevertheless, not all engagement activities are equally effective for multimedia learning. For example, Karich et al. (2014) conducted a meta-analysis of learner control in educational technology research, based on 18 studies (k = 25 comparisons and a total N = 3,618 participants). The analysis investigated the influence of providing learner control over various features of multimedia lessons, including the pacing of the visualizations, ordering of the modules, and opportunities for practice. Overall, the effect of providing these engagement options was of g = 0.05, which corresponds to a small effect size according to the benchmarks by Cohen (1988). This small effect suggests that allowing students to control aspects of the learning material may not be the most robust method of fostering productive engagement with multimedia.

Particularly problematic are the engagement actions that induce high cognitive load and thus leave fewer working memory resources available to learning (see Castro-Alonso et al. this volume-b, Chap. 5; see also Mayer and Fiorella 2014). For example, in an experiment with 144 medicine undergraduates studying stroke symptoms from interactive multimedia, Song et al. (2014) randomly assigned students to four different levels of engagement. Two of the groups involved high cognitive engagement, as they required careful consideration of the answer before either clicking or dragging with the mouse. This careful consideration involved using the mouse to find the location of the blockage causing the stroke. These high cognitive action groups performed significantly lower on a subsequent transfer test than the groups involving less cognitive interactivity, which did not require finding the strokes position (see also Kalet et al. 2012).

Similarly, in three experiments totaling 370 university students (66% females), Stull and Mayer (2007) investigated the instructional effectiveness of using concept maps as aids to learning the reproductive barriers between species. The three studies, which differed in the complexity of the concept maps, randomly assigned students to a learner-generated condition (concepts maps were built by the students) or a multimedia-provided group (concept maps were shown to students). Results across the three experiments consistently showed that the multimedia-provided group outperformed the learner-generated group on transfer tests. As predicted by cognitive load theory, and in contrast to the ICAP framework, generating concept maps required excessive activity, which may have taxed working memory and hindered learning.

In a study by Fiorella and Mayer (2012) college students played an educational game teaching them about how electric circuits work. In the first experiment, students who were provided with a worksheet listing the underlying physics principles taught in the game outperformed students not provided with a worksheet on a subsequent transfer test. However, in the second experiment, participants had to use the worksheet to generate the physics principles on their own while playing the game. Results indicated that only a small group of students who were able to generate the principles accurately benefited from filling in the worksheet, whereas most students could not generate the principles on their own. These findings highlight the potential limitations of engaging students by asking them to generate content on their own.

In all, research on cognitive engagement—and related findings of generative learning and embodied cognition—predict that multimedia lessons that allow higher cognitive engagement, such as by incorporating interactive and constructive activities, will be more effective than lessons allowing less engagement, such as active and passive activities. However, when the multimedia is already highly complex and taxes the necessary working memory to deal with the learning elements, adding excessive engagement activities could be counterproductive (see also Fiorella et al. 2012a).

Thus, a balanced degree of engagement should be provided to the student, particularly to those with a lower capacity of working memory or visuospatial processing. As described next, in addition to these engagement possibilities of the learner, how the multimedia responds to learner engagement plays an essential role in supporting meaningful learning.

6.1.2 Multimedia Feedback

In the broader educational research literature, Hattie and Timperley (2007) defined feedback as information provided by an agent related to the learner’s understanding or performance. In interactive multimedia learning events, the agent is the multimedia, which provides feedback in response to students’ engagement activities. Feedback has also been termed as scaffolding or guidance in the literature on learning from simulations (e.g., Fraser et al. 2015).

There are two broad levels of feedback: feedback about the task and feedback about the processing of the task (Hattie and Timperley 2007). In a review of simulations and educational videogames, Johnson et al. (2017) termed these levels outcome feedback and process feedback, respectively. In the multimedia learning literature (e.g., Moreno 2004; Moreno and Mayer 2007), these levels are commonly called corrective feedback and explanatory feedback (see Fig. 6.1). An example of corrective (lower-level) feedback is hearing a cheerful sound after providing a correct answer in a simulation or videogame (cf. Miranda and Palmer 2014). This level of feedback is the most common but may only lead to surface learning or non-improved performance (e.g., Adam and Vogel 2018).

In contrast, explanatory (higher-level) feedback can lead to deep learning and better performance. For example, a chemistry simulation could show two representations of the same molecule so that when learners manipulate one representation, they see the effects on the other format and can understand their relationship (cf. Barrett et al. 2015). In multimedia research, fostering a more complete and deeper explanatory feedback over the shallower corrective option is referred to as the guided feedback hypothesis (Moreno 2004) or the feedback principle (Moreno and Mayer 2007).

Research on learning from science multimedia among university students supports the feedback principle. For example, Van der Kleij et al. (2015) conducted a meta-analysis of 40 studies (k = 70 effect sizes) to investigate the effectiveness of three levels of feedback in computer learning environments. Many of the studies concerned science subjects (41%) and university or college students (80%). The levels of feedback consisted of: (a) knowledge of results, which solely signaled if the answer was correct or incorrect; (b) knowledge of correct response, which signaled if the answer was correct or incorrect, and also revealed the correct answer; and (c) elaborated feedback, which signaled if the answer was correct or incorrect, revealed the correct answer, and also elaborated on the given information. Overall, the meta-analysis showed that elaborated feedback was more effective than the lower levels, yielding a medium effect size (g = 0.49). More recently, a review by Rivière et al. (2018) similarly highlighted the importance of deep feedback for medical simulations, and Johnson et al. (2017) also recommended providing detailed feedback when designing military and science simulations.

The videogame literature also supports providing more in-depth feedback. For example, Moreno (2004) reported two experiments with a total of 104 psychology undergraduates that learned from a videogame involving the design of plants for different weather conditions. In each experiment, students were randomly assigned to either the explanatory feedback condition (a narration explained the correctness or incorrectness of each answer) or the corrective feedback group (a narration only communicated whether each answer was right or wrong). Both experiments showed that the explanatory feedback groups had higher transfer test scores and gave higher ratings of feedback helpfulness than the corrective feedback conditions.

A follow-up study by Moreno and Mayer (2005, Experiment 1) with a similar botany multimedia videogame also compared explanatory and corrective feedback. In the experiment with 105 psychology undergraduates (70% females), students in the explanatory feedback conditions achieved higher transfer scores and gave better explanations for their choices, compared to participants receiving only corrective feedback. Recently, meta-analyses of videogames for education by Clark et al. (2016) found that including elaborated feedback in the videogames led to greater learning than providing simpler feedback (indicating errors or displaying the correct answers).

Why is explanatory feedback useful? Johnson and Priest (2014; see also Johnson et al. 2017) argue that the primary purpose of feedback is to help students learn from mistakes. As such, explanatory feedback provides complete information that guides students on how to improve their current level of understanding or performance. In contrast, corrective feedback only informs whether students answered correctly or incorrectly, so it does not provide students with a clear direction for improvement. Without this information, students may try to figure out the correct answer themselves, likely taxing working memory resources and hindering performance. This explanation is consistent with cognitive load theory (see Castro-Alonso et al. this volume-b, Chap. 5; see also Sweller et al. 2011), which predicts that explanatory feedback will be more effective because it provides students with necessary support, whereas corrective feedback creates extraneous cognitive load that hinders learning.

In addition to providing insufficient information, feedback that contains unnecessary or irrelevant information can create extraneous cognitive load and be counterproductive for learning (see the coherence principle in Mayer and Fiorella 2014). Consequently, the most effective explanatory feedback is that including only essential information. As with engagement, extra feedback can sometimes be detrimental for learning, especially if: (a) the multimedia is complex, so it already places heavy demands on students’ limited working memory capacity; and (b) the student has a lower capacity of working memory or visuospatial processing, so these systems get overloaded more easily (see also Sect. 6.2.2).

An example of this problematic feedback is provided by L. Lin and Li (2018), who investigated 116 university students (74% females) learning from physics simulations about how objects are affected by phenomena such as friction and gravity. The students were randomly assigned to three feedback groups: (a) control, which was not given any visual feedback; (b) irrelevant feedback, which was shown their individual brain waves of attention after watching each simulation; and (c) irrelevant plus relevant feedback, which was shown the same brain waves plus being prompted to reflect on their learning verbally. The results showed no effects on learning performance or time. However, the ratings of perceived attentiveness and perceived value were significantly lower for both feedback conditions, compared to the control group. Because these formats of feedback were creating additional cognitive load, this processing possibly diverted the students’ working memory resources away from the core learning activities.

Finally, a study by Fiorella et al. (2012b) indicates that the modality of explanatory feedback can also influence cognitive load and significantly impact performance during complex learning. In the experiment, 60 undergraduates (40% females) completed military call-for-fire tasks within a simulated environment and received either visual feedback, auditory feedback, or no feedback. Results indicated that receiving auditory feedback led to the highest performance on a subsequent transfer task. Apparently, the visual feedback competed for cognitive resources with the other complex visual information in the simulated environment. In contrast, auditory feedback allowed students to process the simulated environment and the feedback simultaneously (see also the modality effect in Low and Sweller 2014; see Castro-Alonso et al. this volume-b, Chap. 5).

In all, student engagement and multimedia feedback are the exchanges of information that allow learning from interactive multimedia. Findings across several studies involving university students of health and natural science topics indicate that deeper engagement and feedback tend to promote meaningful learning outcomes. Nonetheless, it is also important to avoid overloading working memory with too much cognitive engagement or unnecessary and irrelevant visual feedback, which could be especially counterproductive for students with low visuospatial processing capacity. These conclusions should be considered when designing any interactive multimedia, including the science simulations described next.

6.2 Simulations

According to Johnson et al. (2017), simulations are models of a real-world system in which the user tests variables to learn how they affect the system. Simulations can provide several advantages for science learning when compared to real laboratory activities, which also test variables in a system.

Concerning these advantages, Triona and Klahr (2003) observed that simulations are preferable to physical laboratories for: (a) lower costs of replication and distribution of the activities, (b) more manageable data collection and thus faster data analysis, and (c) reduction or amplification of spatial and temporal dimensions. Brinson (2015) adds the advantages of: (a) visualizing otherwise unobservable phenomena; and (b) removing confusing physical laboratory details, such as equipment miscalibrations or errors. Also, the time savings of using simulations can be substantial, considering that less dexterity is needed to manipulate and assemble virtual materials compared to real components (Klahr et al. 2007).

Furthermore, Potkonjak et al. (2016) noted that simulations are important for: (a) saving money, as they elude buying reactants and real equipment; and (b) flexibility, as many elements of the simulations can be changed (e.g., parameters, apparatus) to produce novel experiences that would be much more difficult in real settings. Another advantage of simulations, which is also present in videogames, is what Plass et al. (2015) called graceful failure. The feature, also discussed in Triona and Klahr (2003; see also Potkonjak et al. 2016), takes advantage of the unique opportunity simulations give students to fail without consequences. Failing and trying again has less significant consequences with simulations than in real-world settings, encouraging students to take risks, try novel approaches, and learn from mistakes.

Nevertheless, Potkonjak et al. (2016) observed that the graceful failure advantage could also be a drawback. This disadvantage is that students may respond to the lack of real-world consequences by engaging with the simulation carelessly. Another disadvantage of simulations versus physical laboratories, noted by Brinson (2015), is that simulations may not train for science activities in the real world, but only in ideal scenarios. A summary of these advantages and potential disadvantages of simulations is shown in Table 6.1.

Table 6.1 Advantages and (potential) disadvantages of simulations versus physical laboratories
Full size table

6.2.1 Simulations and Science Education

Several reviews (e.g., Brinson 2015; Plass and Schwartz 2014; Potkonjak et al. 2016; Smetana and Bell 2011) have discussed the beneficial instructional effects of simulations and virtual laboratories on different health and natural science disciplines. For example, the review by Smetana and Bell (2011), which included 61 articles of natural science education from the elementary to the university levels, showed that these interactive multimedia tools promoted: (a) science content knowledge; (b) conceptual change; and (c) science process skills, such as visualization, classification, and experimental design. Similarly, Brinson (2015) included six learning outcomes of simulations (and physical laboratories) in his KIPPAS nomenclature: Knowledge & Understanding, Inquiry Skills, Practical Skills, Perception, Analytical Skills, and Social & Scientific Communication. For example, Fig. 6.2 shows a medicine simulation fostering practical skills.

Fig. 6.2
figure 2

A medicine simulation fostering the practical skills of laboratory safety

Full size image

The benefits of simulation-based learning have been documented in studies across a wide range of science areas, including: (a) anatomy (Nicholson et al. 2006), (b) neuroanatomy (Allen et al. 2016), (c) surgery (Keehner et al. 2006; Kostusiak et al. 2017), (d) emergency medicine (Ilgen et al. 2013), (e) nurse skills (Donovan et al. 2018), (f) cell biology (Parong and Mayer 2018), (g) biotechnology (Bonde et al. 2014), (h) chemistry (Blackburn et al. 2019), and (i) geology (Piburn et al. 2005).

For example, the study by Nicholson et al. (2006) illustrates the importance of student engagement in an anatomy simulation. The authors investigated 57 medicine undergraduates learning ear anatomy from either a three-dimensional (3D) interactive simulation or two-dimensional (2D) non-interactive images. The 3D interactive version allowed students to manipulate an ear model across its three axes, as well as to pan and zoom their view of the model. Results showed that the interactive multimedia was a more effective learning tool, arguably because it allowed more productive cognitive engagement by the student. Similarly, Piburn et al. (2005), in a study with 103 geology university students (53% females), reported positive effects of an interactive multimedia lesson on a geospatial test that included items with topographic maps and cross-sections (see Sect. 6.2.2).

In conclusion, simulations are useful tools for learning health and natural science topics. In a review of simulations for secondary science education, Rutten et al. (2012) discussed that moving the field forward should involve adopting a zoomed out perspective that considers the broader educational context. In other words, the design of educational science simulations must consider the role of additional cognitive, motivational, and social factors likely to influence learning. As we describe next, one factor that can moderate learning from simulations is the students’ visuospatial processing ability.

6.2.2 Simulations and Visuospatial Processing

Further information about visuospatial processing abilities and instruments is in Castro-Alonso and Atit (this volume, Chap. 2), and similar instruments are described in Castro-Alonso et al. (this volume-a, Chap. 8) and in Castro-Alonso et al. (2018). In this section, we focus on how visuospatial ability helps to learn from simulations in health and natural sciences.

Several studies within the health sciences, particularly in the context of learning anatomy, have established a link between visuospatial ability and simulation-based learning. For example, in a study by Garg et al. (2001), 146 university anatomy students (50% females) learned from a human hand bones model. They found that 3D mental rotation, a spatial ability measured with the Mental Rotations Test, was a significant predictor of success in the visual anatomy test.

Similarly, in a study with 29 adults (35% females), Loftus et al. (2017) observed that high scorers on the Mental Rotation Test outperformed lower scorers in solving human thorax and ankle anatomy tests with static images. Finally, Allen et al. (2016) investigated 47 medicine undergraduates learning neuroanatomy from both an interactive 3D brain simulation and cadaveric brains. Results indicated that performance on neuroanatomy tests was correlated with scores on a computer spatial ability test that involved imagining 2D sections from complex 3D shapes.

Keehner et al. (2006) examined the role of visuospatial processing in learning from surgery simulations. Forty-four university students completed two pen-and-paper instruments of mental rotation (including the Mental Rotations Test). The authors aggregated the results of both tests into a global score. Then, participants trained for 12 sessions within a virtual reality simulation of an angled laparoscopic surgical task. Results indicated that the simulation enhanced students’ surgical skills, and further, that mental rotation score was correlated to performance (even in the last session). Thus, this study highlights the important role of mental rotation ability in learning from laparoscopic simulations.

Within the natural sciences, biology visualizations and simulations generally demand students’ visuospatial processing (see Castro-Alonso and Uttal 2019). For example, Huk (2006) investigated 106 high school and undergraduate biology students (67% females) learning the structure and function of plant and animal cells from visualizations. Half of the participants also used interactive 3D models of the cells. The Tube Figures Test, a 3D mental rotation test, was used to measure students’ visuospatial processing ability. The results showed that only high spatial students could cope with the mental demands imposed by 3D models, and thus profit from interacting with these sophisticated tools.

Similarly, in an experiment with 112 high school biology students (64% females), Huk and Steinke (2007) assessed participants’ learning about the structures of plant and animal cells from interactive visualizations. Two instructional techniques for presenting the visualizations were compared: (a) close-up, zooming the visualizations; and (b) connecting lines, showing links between the cell structures and verbal labels. Using a median split of the Tube Figures Test scores, students were categorized as high- and low-spatial ability individuals. Results indicated that students with high spatial ability outperformed the low spatial students, especially for the condition with the connecting lines. Overall, these studies suggest that visuospatial ability is important for learning from interactive biology multimedia that demand students’ engagement.

Visuospatial processing also aids learning from simulations and interactive visualizations about other natural science topics. For example, Urhahne et al. (2009) reported two studies totaling 92 chemistry and biochemistry undergraduates (54% females) learning about modifications of carbon from interactive 3D simulations or non-interactive 2D illustrations. Regression analyses revealed that spatial ability was a significant predictor of performance for conceptual knowledge tests, but not factual knowledge tests.

Concerning geology, Piburn et al. (2005) reported a quasi-experiment with 103 university students (53% females) in which the experimental group was given a multimedia with interactive 3D blocks to visualize complex geologic structures. The experimental group was compared to a passive control group that did not interact with the 3D blocks. All participants completed measures of 3D mental rotation (adapted Mental Rotations Test) and mental folding (adapted Surface Development Test). As expected, the treatment with interactive multimedia increased the scores in a geospatial test that included tasks with topographic maps, geologic cross-sections, and perspective taking. Moreover, mental folding (but not 3D mental rotation) was a significant predictor of performance on the geospatial test.

Finally, in three experiments totaling 180 undergraduates, Keehner et al. (2008) investigated the influence of 3D mental rotation on tasks that involved cross-sections of a complex 3D shape with internal branches in different directions. The experiments showed that high mental rotators outperformed lower mental rotators on tasks with the complex shape. These effects were observed in both the interactive version (which allowed rotating and moving the complex shape) and the non-interactive format (which did not allow these generative activities). Also, attempting the tasks was equally effective after studying the interactive or non-interactive versions. In conclusion, these findings provided additional support for mental rotation as an effective multimedia learning aid, and also showed that not all generative or engagement activities help when studying multimedia or simulations.

6.3 Videogames

Videogames have also been termed computer games, digital games, online games, and web-based games (see Cheng et al. 2015). In recent years, research has accumulated showing the effectiveness of these products for enhancing learning (e.g., Wouters and van Oostendorp 2017; see also Tobias et al. 2014). For example, in two meta-analyses comprising 69 studies and more than 6,600 school and university students, Clark et al. (2016) investigated the learning effects of training with videogames on several disciplines, including psychology, maths, literacy, and science (among others). The first meta-analysis (13% science studies) contrasted the learning effects of training with videogames vs. other forms of training. This analysis showed an overall small to medium effect (g = 0.33) favoring videogames.

The second meta-analysis (17% of science studies) compared the learning effects of playing a non-enhanced vs. playing an enhanced educational videogame. The enhancement was provided by guidelines known to benefit learning from videogames, such as feedback, collaboration, and competition. This value-added meta-analysis also showed an overall small to medium effect (g = 0.34) favoring enhanced over non-enhanced educational videogames. These meta-analyses indicate not only that videogames can generally be effective assets for learning various disciplines including science, but also that their design based on established instructional principles can further enhance learning.

Research has provided two positive and two potentially negative outcomes of playing videogames for science education. On the positive side, playing properly designed educational videogames can enhance the learning of topics within the health and natural sciences. Also beneficial is the finding that videogaming can help train visuospatial processing abilities, because these abilities are positively related to science learning (see Castro-Alonso and Uttal this volume, Chap. 3).

On the negative side, playing videogames can sometimes result in wasting time instead of investing it in educational activities. This negative outcome is usually reported for entertainment videogames, not for educational videogames. Another potential negative result connects videogame play, particularly the genre of action entertainment videogames, to aggressive behavior. A summary of these four educational outcomes is shown in Table 6.2.

Table 6.2 Types of educational outcomes for videogame playing
Full size table

Researchers investigating the first adverse outcome argue that devoting time to entertainment-based videogames can reduce time devoted to learning (e.g., Weis and Cerankosky 2010; see also Cummings and Vandewater 2007). Of course, this logic can be applied to any activity that is not directly relevant to learning. In addition, research suggests that not all videogame time competes with learning. For example, in a study involving a large sample of high school students (N > 30,000, ∼51% females, 14–18 years), Hartanto et al. (2018) investigated the correlations between self-reported videogame play and scores on standardized academic tests (including natural science). Results showed small negative correlations in weekday videogaming, but smaller and positive correlations in weekend videogaming. This suggests that there might only be negative effects when videogaming interferes with academic duties (in weekdays), but potentially positive or negligible effects when gameplay takes place primarily on the weekend, at a time when gaming interferes less with educational responsibilities.

The second potential negative viewpoint predicts that playing videogames can lead to aggressive behavior in students (e.g., J.-H. Lin 2013), which should negatively affect learning. However, Ferguson (2010) discussed several empirical and theoretical problems with this perspective. One main problem is that aggressive behavior among the general population has generally decreased, whereas action videogame consumption has rapidly accelerated. Ferguson (2007) provides additional evidence in two meta-analyses relating action videogames to either aggressive behavior or positive outcomes such as visuospatial cognition. The aggressive behavior meta-analysis (k = 21 effect sizes; N = 3,602 participants) showed less than 1% of the shared variance between action videogame playing and aggression. Thus, the hypothesis that action videogaming can lead to aggression was not supported (see also a recent controlled experiment in Hilgard et al. 2019). In contrast, the positive outcome meta-analysis (k = 14, N = 384) showed a 13% overlap of variance between action videogame playing and visuospatial processing. This positive relationship is addressed in later sections (see Sects. 6.3.2, 6.3.3 and 6.3.4). The other positive relationship, which links videogaming to science learning, is discussed next.

6.3.1 Videogames and Science Education

The diversity of research on videogames for science education is growing (Fig. 6.3 shows an example of a biology videogame). As such, a comprehensive review by Boyle et al. (2016) showed that the most popular learning areas for research on educational videogames were health sciences (32% of the studies), followed by natural sciences (17%) and computing (10%). The review also showed that most of the designs for these investigations were quasi-experiments (46%), surpassing true experiments (15%) and correlational designs (10%). In a literature review of videogames for natural science education, Cheng et al. (2015) reported that most of the studies focused on interdisciplinary sciences (47%), followed by biology (31%) and physics (22%). This review also showed fewer examples of university-level education in natural sciences, as only 16% of studies covered this level, compared to 72% dealing with school-level samples. Regarding the seven genres of games reviewed, adventure or role-playing (55%) and simulation (21%) were the two most common for natural science education. This is not surprising, as these types of games include avatars and situations where solving problems and engaging in inquiry activities can lead to meaningful learning. In short, there is ample interest in investigating the effects of videogames on science education across diverse research designs, academic disciplines, and games genres.

Fig. 6.3
figure 3

A biology videogame about DNA translation

Full size image

However, more important than the diversity of research is whether there is empirical evidence showing that videogames are effective tools for learning health and natural science topics. The meta-analysis by Clark et al. (2016), described above, showed positive relationships between videogaming and learning different disciplines, including many science topics. Also, the review by Mayer (2019) showed that in 12 out of 16 experiments, science videogames produced higher learning scores than other science media (e.g., slideshow presentations and web tutorials).

In addition, a number of studies also show positive effects of videogames for university and school students within the disciplines of surgery (Lynch et al. 2010), immunology (Cheng et al. 2014), botany (Moreno 2004; Moreno and Mayer 2005), and physics (Anderson and Barnett 2011). A related variable to science education is the relationship between playing videogames and visuospatial processing.

6.3.2 Videogames and Visuospatial Processing

Arguably, more studies have investigated the relationship between videogames and visuospatial processing than between these games and science education. For example, the systematic review by Cheng et al. (2015) highlighted this focus of research on videogames. The report showed that fewer studies investigated science learning outcomes (14%) compared to cognitive skills (55%), and most studies involving cognitive skills focused on visuospatial processing ability.

There are two types of research investigating the relationships between videogames and visuospatial processing: correlational and experimental (see Table 6.3). The correlational evidence emerges from comparing the visuospatial performance of experienced videogamers versus non-gamers. Finer correlational studies compare different genres of videogames. For example, action gamers are usually compared to non-action gamers, who are players of other genres (e.g., role-playing, strategy, and logical games). These finer comparisons assume that action videogames, mostly first-person shooter games, are more demanding on perceptual and cognitive processes to foster visuospatial processing (see Spence and Feng 2010; see also Dobrowolski et al. 2015), compared to non-action videogames.

Table 6.3 Designs and conditions of research about videogames for visuospatial processing
Full size table

The assumption is generally met, as players with experience in action games tend to show better visuospatial processing abilities than non-action gamers (e.g., Colzato et al. 2013). For example, the meta-analysis by Powers et al. (2013) showed a small-sized effect (k = 37, d = 0.27, p = .001) of videogames from different genres on cognitive tasks related to visuospatial processing. Also, the meta-analysis by Sala et al. (2018) showed an overall small effect (k = 16, g = 0.21, p = .005) of diverse videogames on spatial ability tasks.

In contrast, another meta-analysis by Sala et al. (2018), which only included action videogames, showed a medium effect size (k = 8, g = 0.47, p = .001). Similarly, Bediou et al. (2018) showed a large effect size (k = 27, g = 0.75, p < .001) in a meta-analysis including solely action videogames. These meta-analyses support that playing action videogames may contribute more in developing visuospatial processing ability, compared to playing non-action games.

Of course, the correlational findings do not indicate whether there is a causal relationship between videogaming and visuospatial skill. That is addressed by experimental designs. The typical experimental study compares the gains in visuospatial skill from pre- to post-test of two groups of unexperienced players. The treatment condition receives some amount of videogame training to improve in visuospatial tasks. As with correlational studies, there is research with either action treatment (action videogame training) or non-action treatment (non-action videogame training). As shown in Table 6.3, these treatments are commonly compared to either: (a) a passive control condition that did not train, and was only measured in visuospatial pre- and post-tests; or (b) an active control condition that trained in other activities, such as computer tasks or easier versions of the videogames. In the next subsections, we describe experiments investigating the effects of non-action (Sect. 6.3.3) and action (Sect. 6.3.4) videogames on visuospatial processing.

6.3.3 Non-action Videogames for Visuospatial Processing

Dorval and Pepin (1986) conducted a seminal experiment on the effects of non-action videogames on visuospatial processing. In their investigation, 70 non-gamer humanities undergraduates (47% females) were randomly assigned to a passive control group or the experimental group. Participants in the experimental group completed eight sessions of playing an arcade videogame in which a spaceship had to elude obstacles and shoot enemies in a 3D space. Results indicated that playing the spaceship game improved scores on a mental folding test. In contrast, the control group performed similarly in pre- and post-tests.

In a study by De Lisi and Cammarano (1996), 110 undergraduates (75% females) were randomly assigned to two different videogame training groups. In both conditions, the training involved two playing sessions of 30 min each, separated by 1 week. In the treatment group, the students played a videogame involving the rotation and placement of 3D shapes. In the active control group, the training involved playing solitaire card videogames. Results showed that the 3D shape training was significantly more effective to increase performance in the pen-and-paper Mental Rotations Test, as compared to the solitaire training.

More recently, the meta-analysis by Powers et al. (2013) showed a medium effect (k = 77, d = 0.43, p = .001) and the meta-analysis by Sala et al. (2018) reported a small effect size (k = 37, g = 0.21, p = .001) of non-action videogame training on visuospatial processing. These overall effects considered studies using different visuospatial tests, as well as different non-action videogames. However, much of the literature relating non-action videogames to visuospatial instruments has concerned Tetris™, the videogame that involves rotating 2D abstract shapes.

The research involving Tetris has yielded mixed results. In one experiment by Okagaki and Frensch (1994), 57 psychology undergraduates (51% females) inexperienced with Tetris played the game for 6 h across 12 sessions. These students were compared to a passive control group. All participants completed three pen-and-paper tests of visuospatial processing: (a) the Cube Comparisons Test (mental rotation with 3D shapes), (b) the Card Rotations Test (mental rotation with 2D shapes), and (c) the Form Board Test (mental folding). Results indicated that male students who played Tetris improved their scores on the Cube Comparisons Test and the Form Board Test. In contrast, females who played Tetris did not improve in any of the three tests.

Similarly, Moreau (2012) reported the effects of 12 h of total training (16 sessions of 45 min each) with block Tetris-type videogames. In an experiment with 46 university students (52% females), two training conditions were compared: a version using 2D Tetris shapes and a version with 3D Tetris blocks. Both conditions showed pre- to post-test improvement on other visuospatial tasks, but the 3D version yielded the strongest benefits. Specifically, the 2D game improved mental rotations on other tasks with 2D blocks and 2D human bodies but failed on tasks with 3D blocks or 3D bodies. In contrast, videogame training with 3D Tetris blocks improved mental rotations on both 3D and 2D blocks and bodies.

In contrast, other experiments have reported null effects of Tetris training. For example, a study by Sims and Mayer (2002, Experiment 2) investigated 16 female undergraduates with no previous experience in Tetris. The experiment studied improvements in visuospatial processing measured by three common pen-and-paper instruments: (a) the Card Rotations Test (2D mental rotation), (b) the Paper Folding Test (mental folding), and (c) the Form Board Test (mental folding). Participants were randomly assigned to either a passive condition without training or they trained in Tetris for 12 h (1 h sessions across 12 days). Results showed that the Tetris training did not significantly increase scores on the standard visuospatial instruments.

Comparable results were observed in two studies by Pilegard and Mayer (2018) involving undergraduate non-videogame players. The studies investigated whether Tetris training could transfer to six cognitive skills measured on computer-based tests, including four visuospatial abilities: (a) mental rotation with abstract 2D shapes (Card Rotations Test), (b) mental rotation with Tetris and Tetris-like shapes (2-D Tetris rotations), (c) mental folding (Form Board Test), and (d) spatial working memory (Corsi Block Tapping Test). The studies compared two Tetris training conditions to a passive control group. Results revealed that both training conditions—regular Tetris play and Tetris play supplemented with prompts—did not produce significant gains in any of the cognitive abilities measured.

In short, Tetris and other non-action videogames tend to show a small or null influence on visuospatial test scores. Generally, in these experiments the treatment condition was compared to passive controls that did not perform any training. A more rigorous approach is typically used in action videogame training, where the treatment condition is compared to a control group that does receive some comparable training.

6.3.4 Action Videogames for Visuospatial Processing

As reviewed by Spence and Feng (2010; see also Bediou et al. 2018), three genres of videogames are most amenable to train visuospatial processing: action, driving, and puzzle. From this group, most of the research has concentrated on action videogames, particularly on the popular first-person shooter games. Spence and Feng (2010) summarized experimental findings showing that first-person shooter videogames can activate sensory processes, attentional resources, fast coordination of hands and vision, and visuospatial working memory.

By activating these processes, playing action videogames (particularly first-person shooter games) can provide cognitively-demanding experiences that enhance visuospatial processing. For example, Green and Bavelier (2003, Experiment 5) compared two groups of adult non-gamers who received different videogame training for a total of 10 h (1 h in 10 consecutive days). The results showed that participants who trained with an action videogame improved their spatial distribution of visual attention significantly more than participants who trained with a non-action videogame. The same authors reported a later study (Green and Bavelier 2007) in which both correlational (Experiment 1, 20 male undergraduates) and experimental evidence (Experiment 2, 32 adults, 53% females) indicated that action videogame experience provided a positive boost for spatial resolution of vision. In other words, action videogame playing could help people discriminate smaller distances between visual targets and distractors.

Furthermore, Feng et al. (2007, Experiment 2) investigated the effects of 10 h of videogame training (in sessions of 1–2 h) on 20 undergraduates (70% females). Participants were randomly assigned to either play an action videogame or a non-action game. Results indicated that only training with the action videogame improved the scores of the spatial attention and mental rotation tests.

Similarly, Sanchez (2012) investigated the effects of two videogame training conditions on visuospatial processing of 60 university students (38% females). All participants completed tests of 2D mental rotation (Card Rotations Test) and mental folding (Paper Folding Test). During a 25-min training phase, half of the participants played an action first-person shooter game, whereas the other half played a verbal word-making game. The action training group outperformed the verbal-training group on the Mental Rotations Test, but there were no differences on the Paper Folding Test. Thus, visuospatial training with a first-person shooter videogame boosted 2D mental rotation performance but not mental folding.

Finally, a study by Blacker et al. (2014) involving 34 male undergraduates, compared the effects of 30 h (1 h per 30 days) of training with either an action game or a simulation game. Only the action training group significantly improved performance on a measure of visual working memory capacity. However, the effect of action training was smaller for a test of visual working memory precision, and there was no significant performance increase on a dual visuospatial task of working memory (Symmetry Span). Hence, action videogame training was effective for the less-demanding visual working memory tasks, but not for the complex dual task.

Overall, the studies reported above indicate rather positive outcomes of playing action videogames on various visuospatial processing tests. Recent meta-analyses have aggregated these findings. Although Bediou et al. (2018) observed a medium-sized effect (k = 28, g = 0.44, p = .020), Sala et al. (2018) reported no significant positive effect (k = 22, g = 0.12, p = .248) for action videogame training on cognitive tasks related to visuospatial processing. Taken together, there is some evidence that playing action videogames can enhance particular types of visuospatial skills such as 3D mental rotation.

6.4 Discussion

Among the diversity of instructional multimedia tools for the health and natural sciences, we focus in this chapter on simulations and videogames, as they allow students the highest levels of interactivity. We described interactivity as the exchange between the students’ cognitive engagement and the multimedia’s feedback. We followed the ICAP framework to describe student cognitive engagement as high-level (interactive and constructive) or low-level (active and passive). Similarly, feedback provided by the multimedia also ranges from high-level (explanatory) to low-level (corrective). In general, higher levels of engagement and feedback produce more meaningful learning in multimedia settings. However, as predicted by cognitive load theory, excessive engagement or feedback can be counterproductive, particularly for students with low visuospatial processing capacity. Because these students have fewer working memory resources to deal with the visualizations of simulations and videogames, they have problems simultaneously processing the learning elements plus engagement or feedback information. In short, a balanced degree of engagement and feedback should be provided to students.

Concerning simulations, we described their advantages over more traditional physical laboratory activities, although a couple of potential disadvantages were also acknowledged. We also provided evidence from various science disciplines (including anatomy, surgery, biology, and geology) supporting effective learning of these topics from simulations and virtual laboratories. Finally, we addressed that visuospatial processing abilities, such as mental rotation, were helpful to learn from simulations and complex visualizations.

Regarding videogames, we provided evidence of their positive effects on health and natural science learning, and for training visuospatial processing abilities. We consider that these two positive outcomes overpower the two potential negative results attributed to videogames (wasted time and aggressive behavior). Although there is accumulating research on the effectiveness of videogames for science learning, most of the studies have focused on the impact of videogames on visuospatial processing. We considered both correlational and experimental visuospatial research, usually comparing non-gamers to gamers of non-action (e.g., Tetris) or action (e.g., first-person shooter) videogames. We described that non-action videogames tended to show smaller effects than action videogames on visuospatial processing skills. We finished by remarking that robust evidence can be obtained by experimental studies that compare the effects of playing action videogames to the experience of performing similar activities (e.g., playing non-action videogames).

6.4.1 Instructional Implications for Health and Natural Sciences

A first instructional implication is that science simulations and videogames should be designed to foster interactive and constructive engagement by the student and explanatory feedback by the multimedia, rather than lower degrees of interactivity.

A second implication, which follows from the previous, is that higher degrees of interactivity should only provide necessary engagement and feedback, because when this information is irrelevant or unnecessary, working memory or cognitive overload may occur. This is particularly relevant for lower visuospatial processing students and interactive multimedia depicting complex visualizations or topics.

A third instructional implication is to measure visuospatial processing abilities of students, in case remedial actions are needed with the low scorers.

A fourth implication is for science simulations. Learning activities with these multimedia tools should benefit from the advantages of simulations over physical laboratories. For example, the laboratory activities in simulations could imply collecting large datasets, visualizing unobservable phenomena, or employing many reactants and equipment.

A fifth implication concerns science videogames. Acknowledging a potential negative effect of entertainment videogames for inducing a waste of time, educational science videogames should not be promoted to replace other meaningful learning activities.

The last implication is that action videogaming should be encouraged, chiefly those that do not induce aggressive behaviors. They could be beneficial to train visuospatial processing abilities in those students with low performance.

6.4.2 Future Research Directions

Future research should explore ways to design science multimedia that strike a balance between fostering high cognitive engagement (as emphasized by the ICAP framework) and managing learner’s limited cognitive resources (as emphasized by cognitive load theory). If the multimedia is too cognitively demanding, such as when the learning material is highly complex, then adding more engagement or generative activities could be counterproductive (cf. Chen et al. 2018). Future research should systematically evaluate multimedia environments of different complexities and levels of engagement to test predictions derived from the ICAP framework and cognitive load theory. Furthermore, motivational, emotional, and behavioral processes should be examined for a more complete picture of the complex processes involved in learning from science multimedia (see Park et al. 2015; Plass et al. 2014; see also Fraser et al. 2012, 2015).

As visualizations and multimedia do not impact science and other disciplines (e.g., math and technology) to the same degree (e.g., Castro-Alonso et al. 2019b), future research on simulations and videogames should focus on a particular subdiscipline (e.g., biology, chemistry, technology) or compare their differences.

In addition, future endeavors should consider other students’ characteristics in addition to visuospatial processing. For example, sex is a possible candidate (see Castro-Alonso and Jansen this volume, Chap. 4), as it appears to moderate the experiences of learning from visualizations (e.g., Castro-Alonso et al. 2019b; Wong et al. 2018), using computers (e.g., Drabowicz 2014), and playing videogames (e.g., Terlecki and Newcombe 2005).

From a methodological standpoint, future research should continue to increase its standards of rigor. Fiorella and Mayer (2018) observed that comparisons with well-matched control groups are those that follow a value-added methodology. For example, a value-added approach for simulations would go beyond comparing a simulation (experimental group) to other educational material (control group), in order to contrast two simulations that only differ in one feature or variable (see also Castro-Alonso et al. 2016). For videogames, as described by Mayer (2015), this research should follow sound research methods, such as including well-matched active control groups and randomly assigning participants to the treatment and control groups.

Also, Mayer (2017) proposed three core components of a research agenda applicable to interactive multimedia simulations and videogames: (a) investigating emerging technologies, such as virtual reality, and mobile phones; (b) conducting research in real classroom settings, which provide more ecological validity than common laboratory studies; and (c) testing the effects of interventions across longer durations, which provide more robust implications compared to shorter lab-based interventions.

Finally, beyond the scope of this chapter (and book), other visuospatial processing abilities could be considered, such as dynamic spatial abilities (e.g., Sanchez and Wiley 2014) or environmental spatial abilities (e.g., Kozhevnikov et al. 2013). Similarly, research should investigate more direct relationships between particular visuospatial abilities and particular interactive science tasks (see Castro-Alonso et al. 2019a; see also Section 3.5 in Fiorella and Mayer 2018).

6.4.3 Conclusion

Simulations and videogames allow students the highest levels of information exchanges, in which engagement from the students is followed by feedback from the multimedia. Although this high interactivity is usually positive, excessive engagement and feedback can also be negative, especially for low visuospatial capacity learners, as they cannot process the learning elements plus the extra interactive features. Properly designed simulations and videogames, which find a balance in the degree of interactivity, have proven to be effective instructional tools for topics about health and natural sciences. In addition to being effective for science learning, these multimedia products have also shown a positive relationship with visuospatial processing. Most of this research has focused on videogames (e.g., action videogames) and their positive effect on training visuospatial skills. Future research will help in providing greater details on the type of visuospatial processing ability most needed for certain tasks or topics involving science simulations and videogames.