The use of multimedia in teaching science is ubiquitous. It has become increasingly easy to use technology to present and manipulate information in a variety of auditory and visual formats such as animations and simulations. Frequently, the use of multimedia is associated with an implicit assumption that the introduction of educational technology is desirable and beneficial in its own right. That assumption may not always be valid. Human cognition has evolved over many generations and the same cognitive architecture is required to process information whether or not educational technology is used (Sweller in press-a). Although many educators have welcomed advances in technology to create new opportunities for student learning in science, in the absence of reference to human cognitive architecture, the introduction of multimedia instructional procedures runs the risk of having negative rather than positive effects (Mayer et al. 2005).

The design of multimedia instructional materials should be founded on cognitive theory and associated research into instructional design procedures (Deubel 2003; Mayer 2010). In the current work, cognitive load theory is used to generate experimental hypotheses. The theory uses our knowledge of human cognitive architecture in the design of instructional procedures. It can inform us when and how to use auditory and visual information, the concern of the experiments of this paper.

Cognitive load theory

Cognitive load theory (Sweller in press-b; Sweller et al. in press) is based on an understanding of the evolution of human cognitive architecture and has used this understanding to expand knowledge of instructional design. The cognitive architecture used by cognitive load theory is based on the same natural information processing system that underlies evolution by natural selection (Sweller and Sweller 2006). Natural information processing systems include a large information store, long-term memory in the case of human cognition and processes to deal with novel information without destroying the information store, handled by a working memory that is very limited in capacity (Miller 1956) and duration (Peterson and Peterson 1959). The capacity and duration limits are vastly expanded (Ericsson and Kintsch 1995) when working memory deals with organised information from long-term memory rather than novel information that may be random in character.

Working memory can be further divided into auditory and visual components both of which are themselves limited in capacity and duration (Baddeley 1999). The total capacity of working memory can be increased by using the auditory processor for language-based information and the visual processor for visual material such as objects or diagrams (Penney 1989).

Cognitive load comprises three components (Sweller 2010). Intrinsic load, which is determined by the intrinsic complexity of the learning materials, cannot be altered except by changing the nature of what needs to be learned or by increasing the expertise of learners. Extraneous load is imposed by instructional designs that unnecessarily increase working memory load and so can be reduced by instructional procedures. Germane load refers to the cognitive load caused by effortful learning due to working memory resources needed to deal with intrinsic cognitive load. Instructional procedures should aim to decrease extraneous cognitive load in order to allow more working memory resources to deal with intrinsic cognitive load.

The working memory load imposed by intrinsic and extraneous cognitive load is determined by levels of element interactivity (Sweller 2010). An element is any information that needs to be processed. If the relevant information has few elements that need to be simultaneously processed in working memory, element interactivity is low. If it has many elements that need to be processed simultaneously, element interactivity is high. Element interactivity that is an essential component of the information generates an intrinsic cognitive load. For example, learning the names of chemical elements has low element interactivity and a low intrinsic cognitive load. In contrast, learning to balance a chemical equation that requires the consideration of multiple factors simultaneously has high element interactivity and a high intrinsic cognitive load. Element interactivity that is due to instructional factors generates an extraneous cognitive load. For example, instruction that incorporates a large number of redundant elements that learners must unnecessarily process, generates a higher extraneous cognitive load than the same instruction with the redundant elements eliminated.

From this analysis, it can be seen that novel information may vary in complexity depending on the number of interacting elements that must be processed simultaneously in working memory in order to be understood. The level of element interactivity determines how the information will be processed and if the number of interacting elements exceeds the number that can be processed by an individual, learning with understanding may be compromised. Cognitive load theory has generated many instructional effects intended to reduce the interacting elements associated with extraneous cognitive load and optimise the interacting elements associated with intrinsic cognitive load. Two effects, the redundancy and expertise-reversal effects, relevant to the current experiments, will be discussed next. A third effect, the modality effect, that has indirect relevance to the current experiments also will be discussed.

Redundancy effect

The redundancy effect occurs when presenting learners with additional, redundant information results in decreased learning (Chandler and Sweller 1991, 1996; Mayer et al. 2001). Unnecessarily increasing the number of elements of information increases element interactivity because novice learners are likely to be unaware which elements are essential and which are redundant and so must use working memory resources to determine the necessity of all elements. The resultant increase in working memory load is likely to impose an extraneous cognitive load that interferes with learning.

An example of the redundancy effect occurs when a self-explanatory diagram is associated with text that re-describes the diagram (Chandler and Sweller 1991). Eliminating the text may result in superior learning compared to presenting the diagram plus text. As another example, Mayer et al. (2001) found that lower transfer performance occurred when instructors added interesting but irrelevant details to a narration or inserted interesting but conceptually irrelevant video clips within instructional material. Redundant sources of information should be omitted completely from the instructional materials for effective learning to occur.

The expertise reversal effect

Whether or not information is redundant does not just depend on the nature of the information. It also depends on the expertise of the learner. Information that is redundant for a more knowledgeable learner may be essential for a novice, leading to the expertise reversal effect (Kalyuga 2005, 2007; Kalyuga et al. 2003). Assume two instructional techniques, A and B. For novices, A may result in better learning than B. With increasing expertise, that difference may reduce, eventually disappear and even reverse, resulting in the expertise reversal effect.

Consider, as an example, students learning science. They may be presented essential information in auditory form. In addition, they may need a visual representation of parts of that information in order to understand it. Students presented both the auditory and visual information should learn more than students just presented the auditory information in isolation. Assume the same information is presented to more knowledgeable learners. For these learners, the auditory information may be intelligible without visual examples. The addition of visual examples may be redundant. Processing the information elements associated with the visual information and associating the visual information with relevant auditory information may result in an unnecessary increase in working memory load. The extraneous cognitive load imposed by the visual information may lead to the redundancy effect when compared to an auditory only presentation.

The modality effect

The modality effect occurs when, for example, presenting a diagram and spoken text increases learning compared to presenting the same diagram and written instead of spoken text (Mousavi et al. 1995). The effect only occurs if both the language-based and object-based information is unintelligible in isolation (Low and Sweller 2005). If learners must process both the language- and object-based information in order to understand it, then it is advantageous to use both the auditory and visual processor with the resultant increase in working memory capacity discussed above. Alternatively, if the language- or object-information is redundant rather than essential for understanding, extraneous cognitive load will be reduced if one or other should be eliminated. In accordance with the expertise reversal effect, whether language- or object-based information is redundant may depend on levels of expertise.

The current experiments combined aspects of these three effects. Learners were presented object-based science information in visual form along with language-based information in auditory form. For novices the visual object-based information may be essential to understanding resulting in improved performance, whereas for more expert learners the visual information may be redundant leading to reduced performance. Comparing the performance of more and less expert learners on the two instructional procedures leads to the expertise reversal effect.

When dealing with audio/visual redundancy, the effect usually is tested by observing the effect of including or excluding the auditory information (Kalyuga et al. 2000). For most materials, the object-based visual information is essential. The current experiments differed from previous demonstrations of audio-visual redundancy in that they tested the hypothesis that object-based, visual information could be redundant when presented in conjunction with spoken information. If so, the redundancy effect should be obtained by eliminating redundant objects that frequently might be considered essential when teaching science. We hypothesised that with sufficient expertise, object-based visual information presented in science classes can be redundant.

Experiment 1

Experiment 1 was designed to investigate the effectiveness of multimedia materials in science education using the topic of magnetism. The primary explanation of magnetism was provided orally but that explanation could be supplemented with visual examples of experiments demonstrating, at appropriate points, various phenomena associated with magnetism. It was hypothesised that the visual presentation of experiments demonstrating the properties of magnets via a computer screen along with an oral explanation of the science of magnetism would be useful for novice learners who had not had any exposure to this topic previously but redundant and so interfere with learning for students who had been exposed previously to this area, leading to an expertise reversal effect.

Most previous experiments that have been designed to present oral and visual presentations simultaneously have placed their emphasis for the primary communication of the content on the visual presentation (Kalyuga et al. 2000; Tindall-Ford et al. 1997). The presence or absence of the auditory component was varied. In this experiment, the oral explanation remained constant with the presence or absence of the visual display constituting an independent variable.

Method

Participants

The participants were students from Year 5 (approximately 11 years of age and in their 6th year of schooling) and Year 6 (approximately 12 years of age and in their 7th year of schooling). These students attended a private, Australian boys’ school and were of similar socio-economic background. They were a heterogeneous group, with students of multiple academic ability levels. Year 5 had a total of 47 students in two classes and Year 6 had a total of 51 students in two classes, totalling 98 students.

The Year 5 students had not previously been taught principles of magnetism in school. Year 6 students had some prior knowledge of the content of the instructional materials presented as they had studied magnetism as a unit of work in the previous year.

Materials and procedure

The experiment was conducted in one class session of 40 min in three stages: instruction, testing and evaluation. Before the instruction stage, all students were seated together to hear the information required for the session. Students from each class were randomly assigned to one of two groups: auditory plus visual presentation or auditory only presentation. Four students in the Year 6 auditory only group had to leave before or during the post-test and were excluded from the experiment. All students sat at individual computers wearing headphones as the auditory and visual materials were presented simultaneously.

There were two sets of instructional materials, one for each condition. One set consisted of a multimedia presentation, visual and auditory, of a series of science experiments on the topic of Magnetism. The visual presentation consisted of several experiments that demonstrated the fundamental facts and concepts of magnetism. They showed the teacher’s hands manipulating magnets to demonstrate the concepts depicted in Table 1.

Table 1 Facts and concepts of magnetism covered in the presentation
Full size table

The auditory presentation consisted of a full explanation of these experiments and the related concepts they were demonstrating as they appeared on the screen in the visual materials, using scientific language and reasoning appropriate to this level of learner. When presented simultaneously, the two sets of information, visual and auditory, corresponded appropriately.

The second set of instructional materials consisted of the auditory format alone with the students being required to listen to the information through individual headphones. They did not have any visual presentation of the science experiments. Both formats lasted 20 min as determined by the auditory presentation and the students were not able to stop or replay any part of the presentation nor were they able to take any notes.

At the conclusion of the presentations, every participant was required to complete a written post-test. This post-test consisted of two types of questions. Twelve questions, requiring memory of the factual content and referred to as factual retention questions, assessed the participants’ ability to remember factual information given in the presentation. Participants were required to insert remembered facts into spaces to complete sentences correctly or to select the correct factual response from a list of four options. Eighteen questions required the participant to demonstrate understanding of the concepts of magnetism and are referred to as transfer questions because they required application of what had been learned in a different context. These transfer questions tested the participants’ level of understanding of the concepts of magnetism and their ability to transfer and apply this conceptual understanding to different question formats or problem scenarios, or to diagrammatic representations of the concept. All questions were equally weighted and were assessed as either correct or incorrect. See Table 2 for examples of factual and transfer questions.

Table 2 Examples of post-test questions for Experiment 1
Full size table

The participants were given 20 min to complete this post-test. They were instructed to raise their hand when they completed the test to signal to the teacher to collect their test papers.

Results and discussion

The dependent variables under analysis were the students’ scores from the post-test, separated into scores from the factual retention questions, a score out of a possible 12, and scores from the transfer questions, a score out of a possible 18. Means and standard deviations for transfer scores and factual scores for both Year 5 and Year 6 are displayed in Table 3.

Table 3 Experiment 1: Means and standard deviation for post-test scores
Full size table

Factual retention data were subjected to a 2 (Years) × 2 (instructional conditions) analysis of variance (ANOVA). Results indicated a significant advantage for Year 6 over Year 5, F(1,94) = 10.96, MSE = 3.09, p < 0.01, η2partial = 0.104. However, there was no instructional effect F(1,94) = 1.57, p > 0.05, nor was there a Year by Instruction interaction, F(1,94) < 1.

Data from the transfer questions were also investigated with a 2 (Years) × 2 (instructional conditions) ANOVA. Once again there was a significant advantage for the Year 6 students, F(1,94) = 10.34, MSE = 7.18, p < 0.01, η2partial = 0.099 and no difference between instructional conditions, F(1,94) < 1. However, there was a significant Year by Instruction interaction, F(1,94) = 6.49, p < 0.01, η2partial = 0.065. A simple main effects test indicated a significant difference between instructional groups for the Year 6 students with the auditory only group producing higher transfer scores than the dual-modal group, F(1,49) = 7.07, p < 0.01. The Year 5 students, in contrast, did not show this significant difference between instructional groups, F(1,45) = 1.13, p > 0.05. As can be seen from the means, while Year 6 students benefited from not viewing the experiments in the presentation, the difference between means for Year 5 students has reversed. These results demonstrate an expertise reversal effect as defined by cognitive load theory (Kalyuga et al. 2003). While the interaction is disordinal, the simple effects tests indicate that it is primarily due to the visual information being redundant for the Year 6 students.

The significant Year by Instruction disordinal interaction on the transfer questions indicates that prior knowledge of a topic could influence the effectiveness of a multimedia presentation if both visual and auditory formats are presented simultaneously. For the Year 5 students, who had limited or no prior knowledge of the topic of magnetism, there might be a benefit in viewing the experiments demonstrating the concepts of magnetism. Without the visual images to support the auditory explanation, these students, with their limited knowledge of the concepts of the topic, may have had difficulty understanding the scientific explanations given by the auditory presentation. More importantly, based on the current results, the visual information was clearly redundant for the Year 6 students.

The interaction was obtained on the transfer tests only with no effects on the factual questions other than those due to differences in expertise. The most probable reason for the failure to obtain effects on the factual questions is that in order to use a concept or procedure in a new context such as an application, it needs to be learned and automated to a sufficient extent to be manipulated easily in working memory. Answering factual questions may require minimal or no manipulation of information in working memory allowing such questions to be answered equally as well irrespective of the extent to which they have been automated.

Experiment 2

Experiment 2 was an attempt to replicate the results obtained in Experiment 1 using a different science topic. It was, therefore, designed to test the hypothesis that instructional materials that were presented in a single mode would be superior to those presented in two modes if the students had prior knowledge of the concepts and content of the instructional materials. The materials for this experiment demonstrated and explained some of the principles on the topic of light. This topic was selected as it, too, had been a topic previously studied as a unit of work by the Year 6 students and had not been studied in any formal way by the Year 5 students. Experiment 2 also included participants from Year 4. Like the Year 5 students, they had not studied the topic of light in any formal curriculum unit at school.

Method

Participants

The participants were students from the same school as those for Experiment 1. They were in Year 4 (approximately 10 years of age and in their 5th year of schooling), Year 5 (approximately 11 years of age and in their 6th year of schooling) and Year 6 (approximately 12 years of age and in their 7th year of schooling). They were a heterogeneous group with students of multiple academic ability levels. Year 4 had a total of 49 students in two classes, Year 5 had a total of 50 students in two classes and Year 6 had a total of 48 students in two classes, totalling 147 students.

The Year 6 students had some prior knowledge of the content of the instructional materials presented as they had previously studied light as a unit of work. The Year 4 and Year 5 students had not been taught the principles and concepts of light as a curriculum unit and were classified as novice learners.

Materials and procedure

The procedure for this experiment replicated that for Experiment 1. As the Years 5 and 6 students had participated in the previous experiment, the introduction was only a revision of important procedural points such as, no sections of the presentation could be repeated, no notes could be taken, and no questions were to be asked during the testing phase.

The participants were randomly assigned to groups. This meant that some participants repeated their previous instructional conditions and some were presented the alternate instructional condition. This procedure was explained to the participants. As this was the first time that the Year 4 students had been participants in an experiment, they were given a full explanation of the procedure as given in Experiment 1 for the Year 5 and the Year 6 students.

The visual presentation for Experiment 2 consisted of a series of practical experiments and demonstrations that indicated the fundamental facts and concepts of light. They showed the teacher’s hands manipulating sources of light such as torches, candles and sunlight reflected from mirrors or shone through a magnifying glass to demonstrate the principles and concepts listed in Table 4.

Table 4 Facts and concepts of light covered in the presentation
Full size table

Replicating Experiment 1, the auditory presentation was a full explanation of these experiments and the related concepts and facts they demonstrated as they appeared on the screen, using scientific language and reasoning appropriate to this level of learners. When presented simultaneously, the two sets of information, visual and auditory, coincided appropriately.

For this experiment, at the conclusion of the presentations, the participants were required to insert remembered facts into spaces, to complete sentences correctly, or to select the correct factual response from a list of four options for 17 factual retention questions. The participants were also required to demonstrate understanding of the concepts of light as presented and be able to transfer this understanding to novel contexts, including diagrammatic representations and problem scenarios in 22 questions. See Table 5 for examples of factual and transfer questions. Unlike Experiment 1, some of the questions in this post-test were allocated 1–2 marks depending on the level of explanation given by each participant. For example, Question 7 asked: What is the difference between a concave lens and a convex lens? Explain the use of one of these lenses. This question required a factual answer for the first section. The second section, however, required the participant to demonstrate an understanding of the differences between the lenses, which could include the structure of the lenses plus various uses of these in everyday or scientific contexts, including novel contexts. Specific criteria for each extended answer were determined before testing.

Table 5 Examples of questions on the post-test for Experiment 2
Full size table

Results and discussion

Factual retention data were subjected to a 3 (Years) × 2 (instructional conditions) ANOVA. See Table 6 for means and standard deviations for each Year group. Results replicated those for Experiment 1, indicating a significant advantage for Year 6, F(2,141) = 25.64, MSE = 7.19, p < 0.001, η2partial = 0.267. Again, there was no instructional effect F(2,141) = 1.46, p > 0.05, nor was there a Year by Instruction interaction, F(2,141) < 1 for the factual questions.

Table 6 Experiment 2: Means and standard deviation for post-test scores
Full size table

Data from the transfer questions were also investigated with a 3 (Years) × 2 (instructional conditions) ANOVA. Again, Year 6 students had a significant advantage over the Year 4 and Year 5 students, F(2,141) = 6.28, MSE = 15.88, p < 0.01, η2partial = 0.082, with no difference between instructional conditions, F(2,141) < 1. More importantly, replicating Experiment 1, the results also indicated a significant Year by Instruction disordinal interaction, F(2,141) = 5.27, p < 0.01, η2partial = 0.07. A simple main effects test indicated a significant difference between instructional groups for the Year 6 students with the auditory only group producing higher transfer scores than the dual modal group, F(1,46) = 8.99, p < 0.01. Comparing this result to Year 4, F(1,47) < 1 and Year 5, F(1,48) < 1, it is clear that the learning of the Year 6 students was negatively affected by the additional visual input. The means and standard deviations are shown in Table 6.

These results confirm the results in Experiment 1. Learners with prior knowledge of a topic may be disadvantaged and may even be hindered in their learning, underperforming in assessment tasks if a dual-mode presentation of visual and auditory information is used. In contrast, the novice learners of both Year 4 and Year 5 did not suffer a similar disadvantage and based on the disordinal interaction, might have benefited from seeing the science equipment demonstrating the unfamiliar science words and concepts. As was the case for Experiment 1, the effect only was obtained for transfer rather than factual questions.

General discussion

Technological advances permit us to organise instructional procedures in ways that previously were difficult or impossible. As an example, we can readily mix auditory and visual information and present that information without a direct human presence. Such educational technological innovations can provide considerable benefits but there are also considerable dangers. If educational technology is not adapted to the human cognitive system, we run the risk of introducing novel procedures that inhibit rather than facilitate learning.

Providing learners with auditory or visual information, or a combination of both, can be highly beneficial but the circumstances in which a benefit is obtained depends on human cognitive factors. Some of those factors were manifest in the current work. Presenting learners with audiovisual information frequently requires technology and is frequently beneficial. Specifically, and possibly counter-intuitively for some, we hypothesised that while the presence of visual objects might by important for novices in a domain, those same objects might be redundant for more knowledgeable learners. Redundancy imposes an extraneous cognitive load that interferes with learning. If visual objects, normally considered essential in science, are redundant, students will learn more from an auditory presentation alone rather than an audiovisual presentation. In this manner, cognitive load theory was used to hypothesise when audiovisual presentations might facilitate and when they might retard learning.

It needs to be noted that results from previous experimentation on the modality effect have provided evidence to suggest that dual-mode instructional presentations increase available working memory capacity by combining the capacity of both the visual and auditory memory channels. In those experiments, the auditory (spoken) and visual (e.g. graphical) information sources were both essential in order to understand the information. In the control groups, the spoken information was replaced by visual (written) information along with the graphical information. The modality effect was obtained when the audiovisual information proved superior to the two sources of visual information. The current experiments differ in that for the more knowledgeable learners, the audiovisual information was redundant compared to an auditory (spoken) presentation only rather than consisting of two sources of essential, visual information. Comparing audiovisual information with audio information only, tests for the redundancy rather than the modality effect because the possibly redundant visual information is eliminated.

Our results demonstrated that for less knowledgeable learners, an audio presentation might be facilitated by the inclusion of additional, visual information. At the very least, the visual information clearly did not have negative effects. Clearly, that same visual presentation had negative consequences for more knowledgeable learners. These results were predicted based on cognitive load theory. If we assume that the video information was positive or neutral for less knowledgeable learners but redundant for more knowledgeable learners, we can predict an expertise reversal effect, an effect that was obtained in both experiments.

These results indicate that the most commonly demonstrated form of audiovisual redundancy in which the elimination of spoken information increases the effectiveness of visual information, is not the only form. Under some conditions, learning is facilitated by the elimination of visual, object-based information, a result not previously obtained as far as we are aware. Given the emphasis on object-based information in science practical classes, it may be important to indicate that at least under some conditions, learning can be facilitated by the use of spoken information only. The fact that technology can allow information to be presented in visual as well as auditory form may not justify the inclusion of visual, object-based information under all conditions. Sometimes, the presence of objects may interfere with rather than facilitate learning.

While cognitive load theory was used to predict the obtained results, it is always possible that those results can be explained by different factors. We do not discount the possibility of alternative explanations. Nevertheless, any alternative explanation needs to account for the fact that identical instructional procedures can reverse in their relative effectiveness depending on levels of expertise. Cognitive load theory is able to provide an explanation for this disordinal interaction. Alternative explanations will need to be tested, if they can be formulated.

One possible alternative explanation for the redundancy effect obtained in both experiments with more knowledgeable students is a motivational one. It could be argued that learners presented with information that is redundant, are less inclined to fully process it resulting in reduced test scores. From an a priori, theoretical basis, we know that redundancy increases the number of elements that learners must process. From an empirical basis, while the current data did not include independent measures of cognitive load, evidence that redundancy increases cognitive load has been obtained on many occasions (e.g. Chandler and Sweller 1996). Nevertheless, we acknowledge the current data do not eliminate a motivational explanation.

From an instructional perspective, the expertise reversal effect in general and the current results in particular, introduce a conundrum. Our results, like most results from randomised, controlled experiments are relative rather than absolute. We know that novices need considerable assistance from visual materials to understand some categories of spoken information and we also know that the same visual information interferes with further learning when dealing with more knowledgeable learners. We have no absolute measures that indicate at what point visual assistance should be phased out, although some attempts to construct suitable tests in mathematical areas have been made (Kalyuga and Sweller 2004, 2005). At present, the decision to phase out assistance must be left to an instructor. That situation is likely to persist until measures are devised indicating the absolute as opposed to relative levels of expertise at which assistance no longer is required.