Introduction

Creating visualizations comprises an essential practice in the community of science. Research has started to investigate the role of drawing or creating visualizations to support students’ learning of science. There are at least two important roles documented. First, engaging students in creating visualizations can be an effective instructional strategy to foster their conceptual understanding and representation skills (Ainsworth et al. 2011; Gilbert 2008; Zhang and Linn 2011). Second, asking students to draw or visualize their conceptions in assessments can help probe their understanding through the visual mode and provide information complementary to their verbal expression (Cheng and Gilbert 2009; Kern et al. 2010). Studies indicate positive results when presenting interactive, dynamic visualizations to students for them to learn through observations of visualizations (Frailich et al. 2009; Karacop and Doymus 2013; Kozma et al. 1996; Levy 2012; Schank and Kozma 2002; Varma and Linn 2012; Wu et al. 2001). Recent innovative research started to provide dynamic visualizations in assessments to investigate their effect (Naah and Sanger 2013). Equally important but little investigated is how technology can play a role in assessments that ask students to create their own visualizations. The answer to an even more basic question is also unknown: how students take advantage of drawing technology to visualize their ideas in science.

In this study, we examined how students used a drawing tool to demonstrate their understanding of chemistry concepts that are not easily assessed through paper-and-pencil-written achievement tests. We asked 30 students to use the drawing tool to show their ideas of chemical reaction processes and interviewed them to probe connections between their drawings and verbal explanations. The drawing tool, Chemation (Fig. 1), allows students to construct simple 2D molecular models and flip-book-style dynamic visualizations on handheld computers. The content-specific support provided in Chemation includes only the simplified atom palette and real-time messages showing errors when students try to connect all atoms to become a circle. These supports remain minimal and open to allow students to construct molecular models and animations to represent a chemical process. In our earlier testing trials, we provided evidence that students can easily build and revise molecular models or animations in Chemation (Chang and Quintana 2006). In addition, it is more efficient to use technology to collect and analyze students’ visualizations. Nevertheless, this study did not focus on the technology per se, but revealed a case of how to assess students’ visualizations and understanding of chemical reactions in the context of using the technology. Particularly, how the technology was used to assess new aspects of students’ understanding is discussed.

Fig. 1
figure 1

Main features of Chemation

Full size image

In light of its content advantages, drawing technology can benefit assessments by allowing students to construct dynamic visualizations (i.e., draw a series of frames and play the animations) and providing a visual product for students to refer to and connect to their verbal explanations. For example, in the case of learning chemistry, important concepts such as molecule movements and chemical equilibrium in chemical reactions involve dynamic processes and aspects. As studies point out the possibility that dynamic mental representations exist (Freyd 1987; Hegarty 1992; Hegarty et al. 2003), a drawing tool with animation functions allows students to express their ideas or externalize their dynamic mental images about those chemical processes and dynamic aspects.

However, is allowing students to create dynamic visualizations of science events simply a fancy, novel task with few conceptual merits? Indeed, a reasonable doubt would be that students might learn as well when they can draw static visualizations. Although we found research comparing the effect of presenting dynamic versus static visualizations to students (e.g., Hoffler and Leutner 2007; Rutten et al. 2012; Tversky et al. 2002), we did not find any study discussing the benefits, if any, of encouraging students to create dynamic as opposed to static visualizations. In this study, we explore this issue by observing whether the students are able to create dynamic visualizations of chemical reaction processes and by comparing how conceptually different the students who create dynamic visualizations are from those who only create static visualizations, to discern, if any, the differences between these two spontaneous groups of students. The results shed light on the benefits of engaging students in creating dynamic visualizations in the case of learning chemical reactions.

Students can use Chemation to represent the dynamic aspect of the molecular process in a chemical reaction, such as bond breaking, bond formation, and atom rearrangement. Nevertheless, students can construct either dynamic or static visualizations to represent their understanding of a given chemical problem. For example, a dynamic visualization of a chemical reaction generated by students using Chemation can include the reactants in the first frame, the process of atom rearrangement in the middle frames, and the products in the last frame. When the student starts the “animation” mode, the visualization in Chemation dynamically shows the intermediate process, including how bonds break, atoms rearrange, and new bonds form. In contrast, a student may use Chemation to generate only the reactants in the first frame and the products in the last frame without showing the dynamic process, which is defined by us as generating a static visualization of a chemical reaction. Although the student may use the “animation” mode to show the two-frame visualization, it does not meet the criterion of a dynamic visualization showing the intermediate process of the chemical reaction.

The following research questions were investigated in this study: (1) How well were the students able to use a drawing tool to demonstrate their understanding of chemical reaction processes? (2) What is the added value of students constructing dynamic, as opposed to static, visualizations of chemical reactions at the molecular level? We synthesize the results of the two research questions to reflect on the advantages of using drawing tools and interview techniques to assess students’ understanding of chemistry that would be unlikely to be measured through traditional assessments. Although this observational study did not compare students’ performances through Chemation with those through paper and pencil, the results provide insights into the benefits of assessing students’ chemical understanding through visualizations and engaging them in constructing dynamic visualizations of chemical reactions.

Theoretical and Empirical Background

Students’ Conceptions and Visualizations of Chemical Reactions

Representing matter at the molecular level and interpreting molecular visualizations to connect to observable phenomena are important for students to develop an integrated understanding of chemistry [Gilbert and Treagust 2009; Johnstone 1991; Linn 2006; National Research Council (NRC) 2007]. However, many students find it difficult to visualize chemical reactions at the molecular level (Frailich et al. 2009; Krajcik 1991; Nakhleh 1992; Tasker and Dalton 2008). A major issue is that they often treat the equations describing chemical reactions at the symbolic level as mathematical problems rather than linking them to the molecular level or to observable phenomena (Gabel and Bunce 1994). Students often simply memorize such chemical equations (Chandrasegaran and Treagust 2009), and hence might not understand that a chemical equation represents a chemical reaction involving atom rearrangement and bond breaking and formation (Krajcik 1991; Nakhleh 1992; Tasker and Dalton 2008).

Also, students often confuse a physical change, such as a phase change, with a chemical change (Ahtee and Varjola 1998; Stavridou and Solomonidou 1998). Observations of macroscopic phenomena such as a change in color or the generation of gas products might not reveal whether the phenomenon involves a chemical or a physical change. The explanatory power of chemistry is at the molecular or atomic level (Hesse and Anderson 1992; Treagust and Chittleborough 2001), but studies have found that few students explain chemical phenomena at these levels (Abraham et al. 1994; Hesse and Anderson 1992; Stavridou and Solomonidou 1998). All of the participants (i.e., college and graduate students) in one study were found to be unfamiliar with molecular images of chemical reactions, which in turn made it difficult for them to make sense of the underlying chemical process (Stains and Talanquer 2008).

Studies suggest introducing students to chemistry concepts at the molecular level to help them overcome these learning difficulties and develop adequate understanding (Ahtee and Varjola 1998; de Vos and Verdonk 1987; Driver 1985; Driver et al. 1994; Gabel 1993). Having an adequate understanding of chemical reactions involves making sense of the underlying chemical process, which requires students to make meaning of representations, that is, visualizing the chemical reactions (Gilbert 2008). An ideal chemistry curriculum emphasizes students making links among three levels of representation, the macroscopic, submicroscopic, and symbolic levels (Gilbert and Treagust 2009; Johnstone 1991), through carefully designed curricular sequences that address students’ prior knowledge, cognition, and experience (Driver 1985; Gabel and Bunce 1994; Johnstone 2000). A triplet relationship model by Gilbert and Treagust (2009) and Johnstone (1991) clearly identifies three important links among macroscopic, submicroscopic, and symbolic representations for learning chemistry. A qualitative examination of how students make connections across levels can provide insights into curriculum design to facilitate their understanding of chemistry.

Technology Tools to Support Student Visualizations of Molecular Processes

Several visualization tools are known to promote student understanding of dynamic aspects of chemical phenomena, such as eChem (Wu et al. 2001), Molecular Workbench (Xie et al. 2011; Xie and Tinker 2006), and ChemSense (Schank and Kozma 2002). ChemSense enables students to create and edit text, graphs, drawings, and animations of chemistry concepts (Schank and Kozma 2002). Schank and Kozma found that enabling students to create ChemSense drawings helped them to develop representational competence such as the ability to reflectively construct and use representations to think about, communicate, and explain chemical phenomena (Kozma and Russell 2005). The quality of the produced drawings increased with their number. However, they did not further differentiate between students’ dynamic or static visualizations. Molecular Workbench also has highly interactive features to help students develop visualizations of molecular or atomic concepts (Xie et al. 2011; Xie and Tinker 2006). This program enables students to change the parameters of a visual display to develop and test their ideas, theories, and hypotheses. Both eChem and Molecular Workbench enable students to specify variables of the visualization and to explore interactive visualizations, while ChemSense and Chemation enable students to generate their own visualizations with content-specific supports provided by the tool.

Enabling students to generate visualizations using appropriate tools can support their learning and understanding, which is consistent with calls for scientific practice such as modeling in science classrooms (NRC 2007; Schwarz et al. 2009). These computer visualization tools make it easy for students to create dynamic visualizations that represent their understanding of chemical phenomena. While studies have investigated student-generated self-explanations (Ainsworth and Loizou 2003; Roy and Chi 2005), critiques (Chang and Linn 2013), and static pictorial representations (diSessa 2004; Parnafes 2010; Zhang and Linn 2011), no study has focused on student-generated dynamic visualizations. Research is needed to reveal the added value of enabling students to generate dynamic visualizations of their own. This could be time-consuming, and its effect on learning may be simply equivalent to enabling students to generate static drawings such as those showing the reactants and products of a chemical reaction. As an initial step, this study investigated the benefits of generating dynamic visualizations by observing and analyzing how students use dynamic visualizations to help them perform tasks related to the understanding of chemistry.

Methods

Context and Participants

We interviewed 30 seventh-grade students randomly drawn from a sample of 271 who, prior to this study, had used Chemation in a middle school chemistry unit (for how the students used Chemation during class see Chang et al. 2010). Before taking part in the interviews, the students had completed the 10-week inquiry-based chemistry unit that included 14 lessons during three of which they used Chemation to learn about chemical reactions at the dynamic level. The learning goals related to the use of molecular models in the lessons included students being able to (1) construct molecular models to represent a substance as comprising the same type of atom or molecule throughout and a mixture as comprising more than one type of atom or molecule, (2) use molecular models to explain how a chemical reaction differs from physical processes such as boiling and mixing, and (3) explain chemical reactions and conservation of matter at the molecular level.

Most of the students were from racial/ethnic groups traditionally underrepresented in science, and the socioeconomic status of their families was below the average for the state. Nevertheless, all students were literate in the use of handheld computers since they had used this type of computer in two previous inquiry-based units. In summary, these 30 students had experience using handheld computers and Chemation and had learned chemical reactions but developed various understandings after the instruction. Thirteen of the students demonstrated adequate knowledge and 17 had incomplete or partially adequate knowledge of chemical reactions. In this study, we assessed these students’ visualizations through Chemation and interviews that asked them to transfer their knowledge by generating new visualizations and solving new chemical problems different from what they had learned during the unit.

Procedure and Interview Questions

The interviews consisted of two parts, thinking-aloud tasks and follow-up retrospective tasks, to detect the students’ understanding of chemical reactions through visualization. Van Someren et al. (1994) suggested that this method of combining thinking aloud and retrospection probes students’ thinking processes and understanding. We performed several rounds of revisions by a chemist and multiple science educators to reach agreement on the interview content. A protocol was developed to ensure that each interview followed the same procedure. Each interview lasted about 30–40 min and was videotaped. The video camera was set up to capture the screen of the handheld computer as a student constructed his or her visualization, and to capture the student’s actions during retrospection.

Each student was provided with Chemation during the individual interviews. At the beginning of the interview, each student performed one practice task to practice thinking aloud. Then, the student was asked to create a simple animation before the main task to show or gain fluency in their use of Chemation; this ensured that all of the students knew how to create animations using Chemation. The students who showed unfamiliarity with creating animations using Chemation were allowed extra time and were provided with instruction by the interviewer (the first author) to ensure that they were able to create animations before the main task.

For the main task, the student observed the macroscopic phenomenon of the reaction of hydrochloric acid with calcium carbonate (limestone) rock demonstrated by the interviewer. The selection of this chemical reaction was a trade-off, discussed by an expert panel including a chemist and science educators, based on the facts that (1) it involves everyday life experiences (such as using hydrochloric acid solutions as household cleaners for limestone floors), (2) the students had not learned about this chemical reaction in the chemistry unit, and (3) the reactants have relatively simple molecular structures so that the students would not spend too much time constructing the models, given that the focus of the task was on chemical reaction processes. However, making different types of bonds of calcium carbonate was not supported in Chemation and was also beyond the learning goals of the curriculum. Therefore, in the interviews and analysis, we did not focus on whether the student was able to differentiate different types of bonds of calcium carbonate but on whether he/she understood the chemical reaction processes.

After the macroscopic demonstration, the student was then given the chemical formulas and molecular structures of the reactants and two of the three products, and was asked to use Chemation to visualize the molecular process of the chemical reaction and to think aloud while creating the visualization. Our decision to provide the students with both the chemical formulas and molecular structures of the reactants and two of the three products was because their teacher indicated that given the limited experience in chemistry of seventh graders, the students would not know the molecular structure of calcium carbonate as well as that of many other substances. Not providing the molecular structure at the beginning might have hindered the students’ performance in the visualizations since they would not know the molecular structure to start with.

After the thinking-aloud task, the interviewer asked six questions to probe the students’ solution processes and understanding. Two of the questions asked the students to explain their visualizations: (1) Why did they include a certain number of molecular models? And (2) Why did they rearrange the atoms in the way they did? Three of the questions probed the understanding of the students by requiring them to (1) make a connection between the macroscopic phenomenon and its molecular visualization, (2) draw another visualization to show a mixing process of salt mixing with water, and distinguish between the chemical reaction and mixing process, and (3) predict the unknown product in the chemical reaction of calcium carbonate and hydrochloric acid. One question asked the students for their opinion on the purpose of using animations.

Data Coding and Analysis

All student interviews were transcribed with annotations to include the nonverbal gestures made by the students (e.g., pointing to the molecular model on the paper) and their interactions with Chemation (e.g., clicking the “copy” button). A freeware program called HyperResearch was used to help facilitate data coding and analysis. The coding and analysis process involved three phases. During the first phase, we developed coding schemes for (1) content knowledge that students demonstrated during the interview, (2) visualizations that students created to represent the chemical reaction of hydrochloric acid and calcium carbonate, and (3) interpretation of the visualization to connect to the observable phenomenon and to reconstruct chemistry concepts (Table 1). Interview transcripts were coded based on the coding schemes.

Table 1 Coding categories, definitions, and examples
Full size table

To bring to light the details and nuances perhaps overlooked by the codes, in the second phase of data analysis, the first author generated rich narratives to describe the solution processes demonstrated by each student through iteratively reviewing and summarizing the codes and transcripts. The narratives textually and visually summarized what content knowledge was demonstrated by each student, what dynamism and content-adequacy characteristics the student’s visualization had, and how the student interpreted the visualization to link to the observable phenomenon and to reconstruct chemistry concepts.

During the third phase, we conducted inductive data analysis by iteratively reviewing the interim data from the first and second phases and also the transcripts to search for patterns of the students’ understanding of chemical reactions. We identified from the data that the students successfully, less successfully, or unsuccessfully made four types of connections: drawing on existing knowledge, incorporating dynamic aspects of chemical processes, linking to observable phenomena, and reconstructing chemistry concepts. These connections are centered on students’ visualizations and indicate how well the students used the drawing tool to demonstrate their understandings of chemical reaction processes (Research Question 1). We discuss these connections and the various student performances in the results section.

To answer the second research question of what was the added value of enabling students to construct dynamic, as opposed to static, visualizations of chemical reactions at the molecular level, we need to quantify the qualitative data, so that we can quantitatively compare students who generated dynamic visualizations with those who generated static visualizations. First, the students’ visualizations were categorized as either dynamic or static, based on the directionality and continuity components (for definitions and examples, see Table 1). We then rated the students’ performances on the four types of connections at three levels: adequate (a score of 2), partially adequate (a score of 1), or inadequate (a score of 0). The first author and a second independent rater coded all 30 interview transcripts. Comparison of the codes of the two raters revealed an intercoder agreement of 87 %, and the remaining inconsistent codes were discussed and resolved. We used t tests to investigate significant differences of student performances on the connections between students who generated dynamic and static visualizations.

Findings and Discussion

Connections Made Around Visualizations

We identified four types of connections that the students demonstrated as they used the drawing tool to express their understanding of a chemical reaction. We conceptualize the connections in Fig. 2. We propose that using a drawing tool to assess students’ understanding of chemical reactions has benefits for probing these four connections. In the following sections, we illustrate using typical student cases to show how students may use the drawing tool to make these connections. We discuss each connection in a particular order but do not suggest that it is the only possible sequence. We then discuss the various performances of all the interviewed students, to indicate how well the students performed and to reflect on insights into the use of a drawing tool to probe students’ understanding of chemistry.

Fig. 2
figure 2

Four types of connections made around visualizations of chemical reaction processes

Full size image

Drawing on Existing Knowledge to Form Visualizations

To create a visualization, the students drew on their existing knowledge to form it, including linking to the propositional knowledge such as the definition of chemical reactions, and to the visual information such as the molecular structures of substances. This requires students to cognitively retrieve their existing knowledge of chemical reactions and to use that knowledge to generate their own visualizations of chemical reactions. Our analysis indicates that when required to use the drawing tool to form a visualization of a chemical reaction, all of the students used at least part of their existing knowledge. Here, we discuss a student case to illustrate what we meant by students as partially (i.e., not completely) drawing on existing knowledge. AnitaFootnote 1 demonstrated complete content knowledge: she indicated in the interview that the reactants are “the things to start with,” and the products are “the things to end with,” (reactants and products); she was able to indicate both the macroscopic and submicroscopic definitions of chemical reactions: “in a chemical reaction you make something new” (a macroscopic definition), “which means that in a chemical reaction atoms rearrange to form new substances” (a submicroscopic definition); she further differentiated between a chemical reaction and mixing process: “in a mixture you mix things up and you don’t make anything new”; she also had some idea of conservation of matter: “Atoms can’t just appear or disappear.” It seems that Anita had learned basic definitions of the five chemistry concepts.

When we examined the process of Anita using the drawing tool to create the visualization to represent the chemical reaction, we observed several consistencies between her existing knowledge and visualization which we interpret as her drawing on her knowledge to form a visualization. The consistencies include: in the middle frames, she broke bonds and rearranged atoms since “if you don’t break the bonds then it will not become a chemical reaction,” which is consistent with her definition of chemical reactions: “in a chemical reaction atoms rearrange to form new substances.” She also kept all atoms of the molecules throughout her frames, consistent with her knowledge of conservation of matter.

However, Anita’s knowledge of reactants and products did not link to her visualization. When probed in the interview, she was able to define reactants as “the things to start with” and products as “the things to end with.” When she was asked to visualize the chemical reaction, however, she drew the reactants and given products all in the first frame. Then, in the middle frames, she broke the bonds and rearranged the atoms of all the molecules. It seems that her definitions of reactants and products were not activated or applied when she visualized the chemical reaction. It could also mean that those unlinked definitions might only be meaningless reiteration from class or ideas that needed to be elaborated or complemented through the visual mode. Therefore, we determined that Anita partially, as opposed to completely, drew on her existing knowledge to form the visualization. Anita’s case shows that a student may develop complete definitions of chemical reaction verbally but may not be able to use all of the verbal knowledge to form a visualization of chemical reactions.

Incorporating Dynamic Aspects of Chemical Reactions into Visualizations

To form a dynamic visualization, the students need to incorporate the dynamic aspect of chemical reactions into their visualizations, such as the movement of the atoms, and the breaking and formation of bonds between atoms. However, some students took advantage of the functions of the drawing tool to form dynamic visualizations whereas others did not. We use visualizations from two of the students to illustrate what we mean by students’ dynamic versus static visualizations. Matthew created a dynamic visualization of the chemical reaction at the molecular level, while Alice created a static visualization that focused on the reactants and products. Matthew’s visualization comprised 10 frames in Chemation (Fig. 3). The following excerpt is part of Matthew’s thinking-aloud process.

Fig. 3
figure 3

Three main frames of Matthew’s visualization of the chemical reaction

Full size image

Matthew: Here on the paper it says it’s Ca so I click on the atom where it shows all the atoms and I find a Ca…[builds all reactants]… I will move the molecules around to show a chemical reaction appearing. And once they hit each other, all bonds will break and new bonds will be made. So… I am thinking that I should break a bunch of bonds to where there will be two Cls left and a Ca bonded together and I’ll have to have the right types of atoms left to make H-2-O molecules.

When asked to visualize the chemical reaction of hydrochloric acid and calcium carbonate, Matthew was able to focus on the dynamic process of bond breaking and formation, and to visualize this aspect adequately. His visualization demonstrated the two criteria of a dynamic visualization: continuity and directionality. In comparison, Alice created a static visualization that simply focused on the reactants and products, as the following excerpt demonstrates.

Alice: I put a Ca and an O on the screen and then bond them. And then I make a C, then I bond them to the O and the C and then I get an O and bond it to the C. And then I get another O and put it almost under the C and bond it to the C. Then I make hydrochloric acid coming from the top and then I make Cl right next to it and then I bond them. And then I go to step 2, there is gonna be calcium chloride, water, and…

Alice’s visualization contained only two frames (Fig. 4): the first one showed the reactants and the second one showed the products, which were copied directly from the molecular structures provided on the paper. Her visualization of the chemical reaction was static because it lacked the intermediate process, and hence lacked continuity. Later when asked to draw her molecular visualization of salt mixing into water, she was able to dynamically visualize the mixing process. This indicates that rather than the possibility of Alice lacking drawing techniques using Chemation, it is more likely that she had not developed a dynamic view of chemical reactions at the molecular level.

Fig. 4
figure 4

Alice’s visualization of the chemical reaction

Full size image

Linking Visualization to the Observable Phenomenon

During the interview, the students were required to use their visualization to explain the chemical phenomenon and reason for the unknown product. We define this process as linking visualization to the observable phenomenon. The students differed in their success regarding making links between their visualization and the phenomenon. Here, we again use Matthew’s case to illustrate how a student used a drawing tool to successfully link between the visualization and observable phenomenon.

  • Interviewer: How does your animation relate to the experiment?

  • Matthew: It relates because hydrochloric acid is a form of liquid, and liquids move around, so I was moving my molecules around. And during that process a new substance was made, so I had to make a new substance, so I broke bonds and made new bonds to form a new substance.

Matthew was able to associate the models and their movement that he constructed in his visualization with the macroscopic phenomenon. He explained that he moved around the models of hydrochloric acid molecules to represent the form of a liquid. He also deleted the links of the models of the reactants and rearranged the atoms to form new models to represent the new substances. He related these new models as the new substances generated during the experiment in which he observed the occurrence of a chemical reaction. He was able to reason about the unknown product as he was constructing the visualization:

  • Matthew: There will be another thing. It is carbon dioxide, C–O-2.

  • Interviewer: So you found out the unknown product? How did you find it out?

  • Matthew: Because as I had made calcium chloride and H-2-O there are still three other atoms bonded together and there is one C bonded to two oxygen atoms that is C–O-2.

Matthew made a consistent interpretation between his visualization and the macroscopic phenomenon and was able to use the drawing tool to reason the unknown product.

Using Visualization to Reconstruct Chemistry Knowledge

We define reconstructing chemistry concepts as when students make meaning of a visualization that enhances their existing knowledge of chemical reactions. That is, students need to make meaning of their molecular visualizations and connect back to their knowledge of chemistry. We observed that reconstructing chemistry concepts occurred when the students were asked to interpret their visualization and particularly to respond to the interview questions “Why did you include [number] models of calcium carbonate and [number] models of hydrochloric acid molecules?” “Why did you decide to [or not to] rearrange the atoms?” “What’s the difference between a chemical reaction and a mixing process? Use Chemation to show/construct an example of mixing.” We use an excerpt from Kate to illustrate how a student successfully uses her visualization to reconstruct her chemistry knowledge.

  • Interviewer: Did you rearrange the atoms?

  • Kate: Yeah, I broke the bonds.

  • Interviewer: Why did you decide to rearrange the atoms?

  • Kate: I rearranged them cause I… let’s see… I broke the bonds so they can become calcium chloride. I broke the two hydrochloric acid to make the water molecule. And…cause in a chemical reaction bonds are broken and bonds are made.

The example shows that when interpreting her visualization, Kate applied her knowledge of chemical reactions “cause in a chemical reaction bonds are broken and bonds are made.” In fact, we observed that the links between students’ knowledge and visualization are two-way and interactive. Kate’s case shows that interpreting her visualization reinforced her concept of chemical reaction in a positive way.

Summary and Discussions of Student Performances

We summarize the numbers out of the 30 students who successfully demonstrated a connection in Fig. 5 and found that half (N = 15) were able to completely draw on their existing knowledge to form a visualization of the chemical reaction. The other half only partially drew on their existing knowledge, while no students inadequately drew on their existing knowledge. This result indicates a critical role of using visualizations to assess students’ integrated understanding of chemical reactions. Students may develop understanding of chemical reactions verbally but not necessarily visually. Assessing students’ understanding through visualization suggests that the knowledge of half of the students may not be sufficiently elaborated or may not necessarily be linked to their visualizations. Engaging students in constructing visualizations detected gaps in their conceptual understanding which can be addressed in follow-up instruction or remediation. On balance, we did not find any students who demonstrated full inconsistency between their existing knowledge and visualization. This provides evidence that engaging students in constructing visualizations, as in the interview task in this study, prompts them to make connections between existing knowledge and visualization, although some students connected more and others less. This suggests an example of why drawing can promote students’ conceptual understanding (e.g., Ainsworth et al. 2011; Zhang and Linn 2011), since it enables students’ connection between rules or forms of disciplinary representations and personal existing knowledge, and such connection is an important feature of understanding (Olson 2003).

Fig. 5
figure 5

Summary of the numbers of the students who successfully made the connections (out of the total 30 students)

Full size image

Not all of the students were able to incorporate the before-after sequence and intermediate phase of the chemical reaction to generate dynamic visualizations with the aid of Chemation. Of the 30 students, 19 (63 %) constructed adequate dynamic molecular visualizations of the chemical reaction, and 11 (37 %) constructed static visualizations. However, it is unclear from the literature whether there is added value for students to construct dynamic, as opposed to static, visualizations. We address this issue in the next section by comparing the differences between these two groups of students.

Research indicates the importance for students making inferential connections between molecular visualizations and observable phenomena. This is often demonstrated by chemists but not by novice learners (Kozma 2003). For example, Kozma (2003) revealed that chemists are fluent in transforming between multiple representations when thinking about a particular phenomenon, whereas student thinking is constrained by the features of a particular representation. Moreover, chemists use representations to help them think and reason about underlying mechanisms, whereas many students build molecular models without connecting these mechanisms to their observations (Kozma 2003). In our view, connecting molecular visualizations to observable phenomena is a critical part of making meaning of the representations. Whether or not students adequately interpret their molecular visualizations as being linked to the corresponding phenomenon indicates whether or not they have an adequate understanding of both the visualization and the phenomenon. However, only eight of the 30 students were able to fluently make links between their visualization and the phenomenon. Eleven students were able to make some, but not all, links, while the other 11 students showed difficulties in explaining the phenomenon or the reason for the unknown product using their visualization. Studies have found that few students explain chemical phenomena at the molecular level (Abraham et al. 1994; Hesse and Anderson 1992; Stavridou and Solomonidou 1998). The results of the present study suggest engaging students in creating their visualizations at the molecular level to encourage development of molecular explanations. However, in this kind of task, a large proportion of students still need support in making such connections.

The connection between visualization and chemistry knowledge is theoretically considered to be the most difficult to establish since the concepts of chemical reactions involve entities that are not perceptually available. However, this connection is critical for students to learn chemistry (Wu 2002). We found that only seven students successfully reconstructed all five chemistry concepts using their visualization, 18 students successfully reconstructed some of the five concepts, and 5 students failed to reconstruct the chemistry concepts resulting in alternative conceptions. A student may create and interpret visualization and find confusion between the visualization and existing knowledge, which may lead to reconstructing an alternative conception, such as the common confusion between a chemical and physical change at the molecular level. In this study, we found evidence supporting the view that reconstructing knowledge from visualization is a difficult connection to make. However, rather than seeing visualizations that may lead to students’ alternative conceptions, we argue that interpreting their own visualizations helps reveal students’ conceptual pitfalls that require further instruction or remediation. Using the drawing tool as an assessment tool could help probe student difficulties that might not be found through traditional verbal-only methods.

The Benefits of Generating Dynamic Visualizations of Chemical Reactions

The differences between the students who generated dynamic visualizations and those who generated static visualizations were summarized in Table 2. There is no performance difference between these two groups in terms of drawing on their existing knowledge when asked to visualize a chemical reaction using the drawing tool. However, the students who were able to generate dynamic visualizations using the drawing tool outperformed the other group of students on linking to the observable phenomenon and reconstructing chemistry knowledge. Overall the dynamic group outperformed the static group in making connections. We triangulate the results by examining the interview transcripts and narratives to discern the benefits of generating dynamic visualizations of chemical reactions.

Table 2 Mean (SD) and t test results for the students who generated dynamic and static visualizations
Full size table

During the students’ interviews, we observed that their generation of dynamic visualizations was critical to mediate their linking to the phenomenon and to reconstruct their knowledge of chemical reactions. The following examples from two students illustrate our observations: one generated a dynamic visualization while the other generated a static visualization.

When asked to draw a visualization showing the chemical reaction and to conduct thinking aloud, Terry successfully generated a dynamic visualization and spontaneously linked to the phenomenon “The mystery bubbles are CO2.” When later probed by the interview questions, Terry was able to use her dynamic visualization to reason the unknown product, as the following excerpt demonstrates.

  • Interviewer: How does your animation relate to the experiment?

  • Terry: The animation shows how the bonds are breaking and making something new.

  • Interviewer: Did you have an unknown product in the experiment?

  • Terry: Yes, I found a new one, and it’s C–O-2.

  • Interviewer: How do you know?

  • Terry: because, like…in the beginning you have these right here [the reactants on the screen], then you ended up with these right here [the last frame]. You started with one, two, three, four, five, six, seven, eight, nine molecules [sic], but right here you only have…[counting] six. So I took the ones right here and connected them, and I have three left over, and then I counted six plus three and that’s nine. So I connected those three as C–O-2.

The middle frames of Terry’s dynamic visualization mediated her to reason the atomic constitution of the unknown product. Terry was able to use her visualization as a reasoning tool to make sense of the observable phenomenon that involved a gas being produced by a chemical reaction.

In contrast, Jake created only one frame to represent the chemical reaction of calcium carbonate and hydrochloric acid at the molecular level. He drew models of the reactants and then reconnected the atoms of the reactants to represent the product. He generated the visualization shown in Fig. 6 to demonstrate his understanding that in a chemical reaction, atoms rearrange. However, his atom rearrangement was arbitrary rather than being based on any chemistry principles. His visualization did not show how the bonds break and atoms move in the chemical reaction, and he was unable to explain the unknown product, as the following excerpt demonstrates.

Fig. 6
figure 6

Jake’s visualization of the chemical reaction

Full size image

  • Interviewer: Did you find out the unknown product? Is there an unknown product?

  • Jake: No.

  • Interviewer: Why not?

  • Jake: …well it didn’t have an unknown product because when I mix those, calcium carbonate and hydrochloric acid together…it didn’t come out to something new.

Jake concluded that his visualization did not have any unknown product because “no new atom [sic] was generated.” Jake’s visualization of chemical reactions represents a typical student conception that the product of a chemical reaction is a big molecule, with no insight into the constituent interim phases (Andersson 1986; Krajcik 1991; Zhang and Linn 2011). Such a view of chemical reactions makes it difficult for the student to explain an unknown product related to the chemical phenomena. In this study, we found that around half of the students (55 %) who had static visualizations experienced this difficulty.

When using visualizations to reconstruct chemistry knowledge, considering again the case of Jake, before generating the visualization, he was able to define that chemical reactions involved atom rearrangement, whereas the atoms did not rearrange in a mixing process. However, when he was asked to interpret his visualization of chemical reactions, as shown in Fig. 6, he mistakenly used the visualization to reconstruct his concept of a mixture, as the following excerpt demonstrates.

  • Interviewer: Is it a mixture or a chemical reaction?

  • Jake: It’s a mixture.

  • Interviewer: It’s not a chemical reaction?

  • Jake: No.

  • Interviewer: Why?

  • Jake: Because when I take them back apart I still have both of them. I didn’t lose or gain any atoms.

  • Interviewer: What’s the difference between a mixture and a chemical reaction?

  • Jake: Well, a mixture is where you can take apart two substances and a chemical reaction is when you make something new.

This excerpt exemplifies how a common confusion exists between a mixing process and a chemical reaction (Ahtee and Varjola 1998; Stains and Talanquer 2008; Stavridou and Solomonidou 1998). The definitions that “in a mixture you can take apart two substances” and “in a chemical reaction something new is generated” can be mistakenly interpreted as in Jake’s visualization. Jake thought that because the atoms in his visualization could be reconnected back to where they were originally connected and because “no new atom [sic] was generated,” the visualization he drew represented the formation of a mixture. This reflects that verbal expressions only without visualizations of the processes can have vague meanings for students.

In contrast, students who developed dynamic visualizations had rather more precise definitions that helped them to distinguish between chemical and physical processes. For example, Matthew’s visualization of the chemical reaction (Fig. 3) clearly showed rearrangement of the atoms. When he was subsequently asked to draw a visualization showing a mixing process, his visualization clearly showed that the atoms were not rearranged. The following excerpt demonstrates how he used his two visualizations to explain the difference between a chemical reaction and a mixture.

  • Interviewer: What’s the difference between a chemical reaction and a mixture?

  • Matthew: The difference is that in a mixture you are mixing two substances together but…a chemical reaction is when you mix two or more substances together to make a new substance. In a mixture you only put in two substances together but do not make any new substance.

It is clear that not only was Matthew’s visualization of chemical reactions more elaborated than Jake’s, but also Matthew’s definition of chemical reactions was more precise. Indeed, we observed a dynamic interaction between the students’ visualizations and their understandings of chemical reactions. Jake’s example demonstrates that even if students have established adequate verbal definitions of chemical reactions, without a visualization that contains the details of the process, their existing definitional knowledge may be altered and may result in confusion. Creating and evaluating dynamic visualizations provides concrete examples that help students to elaborate their understanding based on verbal definitions or propositions. The results of this study provide strong evidence for this claim—all students who generated dynamic visualizations were able to either adequately or partially adequately reconstruct their knowledge of chemical reactions.

Conclusions

In this study, we interviewed 30 students about the use of a drawing tool to visualize their understanding of chemical reactions at the molecular level. The tool enabled us to assess the students’ four connections of the integrated understanding of chemical reactions, including how students (1) drew on their existing knowledge, (2) incorporated dynamic aspects of chemical reactions, and made inferential connections (3) between visualizations and observable phenomena, and (4) between visualizations and chemistry knowledge. We found variability in terms of the students’ performances in making these connections. In light of the quality of the students’ performances for the four connections, we rated their performances according to three levels: adequate, partially adequate, and inadequate. A future study may examine the interplay between levels of visualization performance and different levels of understanding.

A triplet relationship model proposed by Gilbert and Treagust (2009) and Johnstone (1991) identifies conceptual links for students to make among macroscopic, sub-microscopic, and symbolic representations to learn not only chemistry but also all the sciences. We extend their model by elaborating on the connections the students make to link their multilevel representations to worlds outside of visual representations, including linking to the observable phenomenon, and to the students’ existing and reconstructed knowledge.

Considering advances in technology that have made it possible for students to easily create dynamic visualizations or animations to represent their understanding of science concepts or phenomena that involve dynamic processes, we were particularly interested in student-generated dynamic visualizations. We found that more than half (63 %) of the interviewed students were able to incorporate the dynamic aspects of chemical reactions into their visualizations and to represent their ideas of how atoms or molecules move and bonds break and form in a chemical reaction. Note that these students had previously learned about chemical reactions at the dynamic level with the drawing tool during a chemistry unit. We did not investigate students who had no experience of any drawing tool. However, despite the experience with the drawing tool, still 37 % of the students generated static visualizations of chemical reactions. With the quantitative and qualitative analyses, we found that a lack of a dynamic view of the chemical reaction process may be associated with the difficulty students face in making inferential connections between the molecular visualization and observable phenomenon (Kozma 2003) or chemistry knowledge (Wu 2002). Important details were missing in the students’ static visualizations. The results of this study provide evidence that students’ understanding of chemistry coevolved with their visualization. Future research can conduct experimental designs to further investigate causal relationships between these aspects.

Research indicates that students experience difficulties in making connections and transformations across observable, molecular, and symbolic levels of chemical representations, which hinder their ability to develop a robust understanding of chemistry concepts (Gilbert and Treagust 2009; Kozma 2003). Our study found that about 75 % of the students had different degrees of difficulty using their visualization to interpret the phenomenon or reconstruct chemistry knowledge. We observed that the difficulty in making these two critical connections led to students’ confusion and alternative conceptions, such as the confusion between a physical and a chemical change (Ahtee and Varjola 1998; Stavridou and Solomonidou 1998). We suggest that curriculum designers address how to support students in making these two connections to foster an integrated understanding of chemical reactions. A future longitudinal study can investigate how students’ experiences with the drawing tool affect their later learning of chemical reactions. Research calls for multiple curricular sequences to guide student learning of chemistry (Driver 1985; Johnstone 2000). A learning progression of chemical reactions seems necessary given the importance of understanding chemical reactions to learn science (Gabel and Bunce 1994).

The results of this study have an implication for instruction. While synthesis studies show how student learning can be enhanced by providing them with external dynamic visualizations to observe and interpret (Hoffler and Leutner 2007; Rutten et al. 2012), in the present study, we identified the benefits of engaging students in generating their own dynamic visualizations of chemical reactions at the molecular level and the underlying processes related to the benefits. Through drawing activities in a curriculum, instructors can learn about students’ ideas and understanding. Moreover, instruction engaging students in generating dynamic, as opposed to static, visualizations may help students develop more integrated understanding, as in this study, we found that the students who generated dynamic visualizations outperformed those who generated only static visualizations in terms of demonstrating more integrated understanding of chemical reactions. Conventional chemistry instruction does not stress the development of students’ dynamic visualizations, but based on the results reported herein, we argue that there should be a focus on enabling students to learn and generate their own dynamic visualizations of chemical reactions processes. For example, a study indicated that some students’ drawings of a dissolving process only reflected what they had seen rather than a cohesive understanding (Kelly and Jone 2007). In this study, we differentiated between students’ dynamic and static drawings and found that students who were unable to generate dynamic visualizations were more likely to show incoherent understanding of a dissolving and chemical reaction process.

The results have three implications for how to assess students’ understanding of chemical reactions through visualizations. First, engaging students in constructing visualizations in assessments helped detect gaps in their conceptual understanding for follow-up instruction or remediation, since asking students to create visualizations provides chances for them to examine or express their understanding through a visual mode. Second, asking students to interpret the visualization they generate can reveal how well they reconstruct their chemistry knowledge or what alternative conceptions they might have. Third, requiring students to visualize the intermediate process of a chemical reaction at the molecular level (i.e., the dynamic visualization of chemical reactions) can indicate how well they develop an integrated understanding of chemical reactions, since we found a positive association between students generating dynamic visualizations and demonstrating coherent connections. We identified these benefits of assessing students’ visualizations in the context of using a drawing tool, which provides technological advantages such as for the ease with which students can revise models and create animations. The identified benefits may also apply to paper-and-pencil visualization assessments, but studies are needed to compare the affordances and limitations of assessing visualizations through different media. For example, a possible important question for future studies to investigate would be whether drawing technology helps more students create dynamic visualizations. Qualitative studies examining interactions between drawing technology and students’ thinking and understanding can contribute to identifying the affordances and features unique to drawing technology that benefit student learning.

One study analyzed 1,833 released items from six large-scale assessments such as Trends in International Mathematics and Science Study (TIMSS) and National Assessment of Educational Progress (NAEP) (Miller and Linn 2013) and found that only 17 % of the items required visual-spatial thinking and that only 2 % assessed the practice of constructing representations. Given that constructing scientific visualizations is an important scientific practice (NRC 2011), more studies are needed to address how to assess students’ visualization performance. This study provides a case of using a drawing tool and interview techniques to assess students’ visualization of chemical reactions. Conceptual and analysis frameworks were discussed as an example to analyze students’ visualizations of chemical reactions. Future innovations include the development of online assessments that require students to draw their ideas and enter their explanations, and automatic graphical analysis and scoring systems for data coding and analysis, to address the possibility of assessing students’ visualizations on a larger scale.