Engagement with authentic scientific practices is critical to learning science and may be especially important for learning about systems (2013; Lehrer and Schauble, 2012). We present here the results from a classroom intervention that used modeling and a conceptual representation to support reasoning about systems. Engaging students in modeling provides opportunity for making ideas visible and available for constructive discussion, which can improve learning outcomes (e.g., Jordan et al. 2013b; Jordan et al. 2014). Clement (2000) has argued that model construction and revision is at the heart of scientific practice, which requires a top-down disciplinary perspective, bottom-up raw observations and data, and a dialectic process that encourages meaning-making. Conceptual representations provide an organizing framework for thinking about systems and can aid in model construction.

Complex natural systems are an important part of the world in which we live and, as such, systems thinking cuts across domains (Yoon 2008) and is recognized as a crosscutting concept in the Next Generation Science Standards (2013). Systems are characterized by hierarchical structures that are composed of subsystems and components, which dynamically interact and exhibit mechanisms and outcomes. When engaging with these systems, novices tend to focus on readily observable and stable structures (Hmelo et al. 2000; Mintzes et al. 1991). There are data to support that the hierarchical organization and complexity, invisible elements, and dynamic processes are particularly challenging for learners to understand (e.g., Feltovich et al. 2001).

Visual representations provide guidance that can help students meet the cognitive demand of reasoning with systems. For example, (Suthers et al. 2003) demonstrated that visual representations influenced discussions and helped to make salient system phenomena and interrelations. Additionally, these representations have been used as scaffolds to help make students’ thinking visible (Derry et al. 2006).

In this article, we will focus on models as visual tools supported by conceptual representations that learners can use to construct explanations. Models can serve as visual representations and have been shown to support systems learning gains (e.g., Schwarz and White 2005; Svoboda and Passmore 2013). Conceptual representations are frameworks that help students organize their ideas; and in the case of our work, the ideas used in model development. With the goal of supporting learning about ecological systems, we used modeling and simulations with a conceptual representation that we termed Components, Mechanisms, and Phenomenon (CMP).

Ecosystems Understanding

We argue that reasoning about ecosystems is critical to both scientific and environmental literacy (Anderson et al. 2008, Covitt et al. 2009). Given the impacts of climate change, ecological and environmental literacy is fast becoming a requisite for informed decision-making (e.g., Jordan et al. 2009). Although data are not abundant, there is evidence that the general public is not well versed in ecology (Magtorn 2005; Puk and Makin 2006; Stone and Barlow 2005).

Because of the features of complex systems, it is not surprising that learners have difficulty comprehending ideas in natural and ecological systems (e.g., Goh et al. 2012; Hmelo-Silver, et al. 2007, Jordan, et al. 2009). Particular to ecology, studies revealed difficulties in thinking beyond linear flow, the macro-level, visible structures, and single causality. For example, students tended to view food chains as a linear structure akin to previously held beliefs about chains, and these ideas were particularly robust (Reiner and Eilam 2001). Hogan (2000) also found linear thinking when considering food webs and system perturbations. Instruction about micro-level, invisible structures and processes such as decomposition seemed to be cornerstone concepts for the broader system (e.g., nutrient cycling; Hogan and Fisherkeller 1996). Students had difficulty tracing molecules in a water-based ecosystem, which likely stemmed from a lack of clarity between macro-level and micro-level function (Covitt et al. 2009). Grotzer and Basca (2003) argue that difficulty in understanding ecosystems stems partly from not identifying the underlying causal structure. Their data support the teaching practice of providing explicit discussion about driving mechanisms. Taken together, these studies suggest that interventions designed to encourage dynamic and multi-leveled thinking about ecosystems are clearly warranted.

Conceptual representations to support learning and instruction

In an effort to design instruction that supports learning about systems, we began with the use of a conceptual representation to organize ideas in hypermedia (Liu and Hmelo-Silver 2009). In a randomized experiment, using the human respiratory system as the domain, we compared learning in middle school students (and then replicated the findings with pre-service teachers) who used hypermedia organized around one representation: functions/phenomena with learning in students who used hypermedia organized around another representation: structures/components. In the former, students navigated to the content with “How” and “Why” questions (e.g., How does air get into the body?, Why do we need oxygen?). In the latter structure-organized hypermedia, students clicked on structures to navigate to the mechanisms, functions/phenomena. The content of the two hypermedia was identical, other than this organization. Students who used the hypermedia constructed around a phenomena-organized conceptual representation demonstrated greater learning of micro-level processes in the system. For example, in the context of the human respiratory system, students using the function/phenomena organization were more likely to mention processes that occurred at the cellular level than students in the structure-oriented condition; consistent with a trajectory towards more expert-level understanding (Hmelo-Silver et al. 2007). Indeed, our later work also supports the notion that a more globally-oriented conceptual representation results in students having broader and more creative ideas (Jordan et al. 2013).

We found in our past work, however, that the use of hypermedia alone did not result in a shift of students’ mental models to dynamic systems models (Liu and Hmelo-Silver 2009). This is likely because the hypermedia provided a static representation of ideas. To make the connections between phenomena and mechanisms more dynamic, we next developed simulations that explicitly made visible the underlying mechanisms behind population dynamics, water quality, and nutrient cycling in the context of aquatic ecosystems. These tools were used as part of a two week middle school science unit. Identical pre- and posttests asked students to draw an aquarium, define terms related to the system, and solve several “what-if” problems related to aquatic systems (see Hmelo-Silver et al. 2015 Appendix). These were coded for the use of structures, behaviors, and functions (described later). We found moderate to large effect sizes for pre- to posttest gains in understanding structures, behaviors and functions and shifts in students’ mental models (Hmelo-Silver et al. 2015; Jordan et al. 2013a; Liu et al. 2007). We posit that this occurred because students were able to engage in productive discussions about mechanisms through their collaborative investigations. Indeed, using Hierarchical Linear Modeling, Liu (2008) found that high quality discourse among students engaging with the simulations predicted improved learning outcomes. High quality discourse including warranting claims and connecting theory and evidence as students used two NetLogo simulations to understand aquarium systems. Using the same data, Hmelo-Silver, Liu, and Jordan (2009) examined contrasting cases of students working with the simulations and demonstrated that the more successful group cycled through different aspects of the conceptual representation (e.g., C, M, P) as they made connections between the simulation objects (e.g., red dots) and what they represented (ammonia molecules). The less successful group did not make those connections and tended to describe the behavior of the simulation (e.g., the ammonia goes up when there are more fish). Together these results suggest that a conceptual representation could serve to guide student inquiry and that simulation models provided a rich context for productive discourse.

With the success in adding simulations to the hypermedia, we sought to add further support to help students make increasingly complex connections across system levels. We therefore engaged students in conceptual modeling using the syntax of the CMP representation as used in the current study. The CMP conceptual representation is intended to support learners in framing systems thinking around a particular phenomenon or ecological pattern (P); encouraging learners to generate or recall plausible mechanisms (M) that may result in the (P); and explore the parts or components (C) that interact to result in (M and P). For example, with an ecological pattern (the “phenomenon” in the current study), we used the CMP conceptual representation in conjunction with a curriculum that uses critical questions that encouraged learners to find and generate evidence in support of mechanistic explanations. This representation works with a suite of technological tools that we developed to teach specifically about aquatic ecosystems. This conceptual representation is a refinement of the Structure-Behavior-Function (SBF) conceptual representation described by Goel et al. (1996) and Hmelo-Silver et al. (2007, 2014, 2015). The SBF representation guides students to broadly consider the relevant structures, observe their behaviors and their functional role in context of a complex system. Because SBF was originally developed as part of artificial intelligence research (Vattam et al. 2011) for reasoning about designed devices, we modified the SBF conceptual representation to the CMP conceptual framework to reflect the mechanistic reasoning of ecosystem learning. We also made these changes to be more consistent with the nature of natural systems (Darden 2006) and because of the pragmatic concerns of the teachers (i.e., issues with nomenclature). We retained the explicit focus on the links between how components work together to drive mechanism in our CMP-based hypermedia (example shown in Fig. 1) and by making mechanisms visible in NetLogo simulations (Wilensky and Reisman 2006). For example, in Fig. 2, the screen shot of the simulation makes macro-level mechanisms related to carbon visible.

Fig. 1
figure 1

CMP hypermedia

Full size image
Fig. 2
figure 2

Example NetLogo simulation

Full size image

Pairing a conceptual representation with the explicit practice of modeling allows learners to externalize their thinking and test ideas; essentially providing a vehicle by which mental models can be externalized and collaboratively discussed. This is consistent with the Yoon et al. (2015) curriculum and instruction framework for teaching about systems that emphasize curricular relevance, cognitive-rich pedagogies, tools for teaching and learning, and content expertise. In particular, these authors stress the use of teaching tools that engage visual and conceptual representations in learning. Using video observations and student interviews, Yoon et al. demonstrated that a curriculum developed using this systems teaching framework led students to engage with science and engineering practices and cross-cutting concepts from the NGSS. Emphasizing learning with models, Lehrer and Schauble (2012) demonstrated the importance of collective participation and discussion about these representations. Model-based representations are particularly important for learning about systems as they allow students to represent multiple levels of organization (Buckley 2000); such as micro to macro levels or levels between. We therefore, designed and used a modeling tool that facilitates CMP (see Table 1). In the example of a student model about a particular phenomenon (Fig. 3), the boxes represent components and the links represent connections between components, with explanations of the mechanisms and sources of evidence in the text boxes on the lines.

Table 1 Framing of the conceptual representation
Full size table
Fig. 3
figure 3

Example CMP model

Full size image

An intervention guided by CMP

We designed an intervention that provided students with opportunities to engage the system and its multiple and interrelated components to help them learn about systems and to deal with the challenges that they have in developing a deep understanding. Our work is grounded in sociocultural theory, which suggests that learning is mediated by tools and artifacts in an activity system. The social learning in this project is distributed across time and media in a computer supported collaborative learning (CSCL) environment (Suthers and Rosen 2011).

As part of the sequence of instructional units, students moved from learning about a closed aquatic ecosystem (aquarium) to increasingly open systems that include a pond and a set of marine problems that are caused by ocean acidification. The intervention involved the framing of the conceptual representation (Table 1) and the curriculum described below.

CMP served as a guide to all activities designed for the following aspects of the curriculum. First, the Phenomenon served as a problem context to motivate student learning. The phenomena in our curriculum included managing life support in a fish tank, dead and dying fish in a pond, thinning oyster shells and unusual swimming in damselfish. This perspective guided design of the hypermedia that are organized using CMP language and from the outcome or phenomenon. For example, the why questions in the pond ecosystem (Fig. 1) treated the healthy pond and biodiversity as phenomena to be explained. In the second unit, this equilibrium was disturbed when students encountered the phenomenon of fish dying in a pond. The curriculum materials provided sources of evidence that included paper materials such as graphs and scientific reports; as well as evidence collected through hands-on investigations and computer simulations. Additionally, students were given worksheets that include scaffolds for student inquiry. Simulations are also used to provide opportunities for students to engage with evidence and mechanisms that underlie the phenomenon at different scales. Last, a conceptual modeling tool was used to help students construct explanatory models in terms of CMP. The conceptual modeling tool allows students to create, note, and link representations with the nodes representing the system components and links representing mechanisms.

We previously found that for students to make system level connections, they must first reconcile the roles of system elements and the scale at which such roles were important (Eberbach et al. 2012). Once enabled, students can then provide mechanistic accounts in their models. Furthermore, this mechanistic reasoning that students learn (M in CMP) transfers to other ecosystem contexts integrating photosynthesis and cellular respiration (Sinha et al. 2013; Hmelo-Silver et al. in press). Additionally, we found that students gained ecosystem content knowledge through this process, which also likely supported mechanistic reasoning (Jordan et al. 2013b). Prior research used pre- and posttest designs without a comparison group, limiting the nature of the inferences about overall effectiveness of the curriculum. In this study, using a small-scale quasi-experimental design, we tested the hypothesis that students whose curriculum was guided by CMP would outperform students in typical science instruction with respect to systems thinking about (1) causal mechanisms and connections to phenomena, and (2) micro-level processes. This study is part of an on-going investigation into collaborative learning where aquatic ecosystems are used to teach systems thinking (Eberbach et al. 2012; Hmelo-Silver et al. 2014).

Methods

Participants

The students described in this study were seventh grade students from a Northeastern United States public middle school. Intervention students participated in an eight-week technology-rich ecosystems unit in their science class. Intervention students were from three classes that were taught by the same teacher. Comparison students, taught by another teacher, were from two classes who were given normal, business-as-usual science instruction. We were not able to observe the comparison classroom but the typical curriculum for this school used a mix of text and hands on activities. Specifically, students in the comparison classrooms looked at biotic and abiotic factors, ecosystem organization, relationships in nature, populations, matter and energy transfer, and biomes. Classroom activities included writing reflections and listening to pod casts. Formative and summative assessments were given to support the content. Sixty-five students completed pre- and posttests in the intervention group, and 47 students completed pre- and posttests in the comparison group.

Classroom context

Prior to the study, the classroom had a physical aquarium installed and maintained for 1 month. The teacher used the Systems and Cycles toolkit in the instruction. The toolkit was installed on laptop computers and students worked in small groups of two to six. Students engaged with the technology in groups followed by class discussions about the aquarium, eutrophic ponds, and oceans, and through construction of cognitive maps that elaborated ecosystem mechanisms such as photosynthesis, cellular respiration, and limiting factors. These mechanisms were taught in an effort for students to comprehend ecosystem phenomena including eutrophication, carrying capacity, and nutrient cycling. Three major units of the curriculum focused on (1) aquarium, (2) pond, and (3) ocean systems. In each unit, students were introduced to a driving problem (i.e., design an aquarium and determine the number of fish in an aquarium, explain fish death in a pond, and identify mechanisms to increase carbon sequestration) and then the students were asked to draw and create models using the modeling tool (Vattam et al. 2011), study phenomena-oriented hypermedia (Liu and Hmelo-Silver 2009), and use NetLogo simulations (Wilensky and Reisman 2006). The driving problem served as the phenomenon to be explained.

The three units extended over several months. To introduce the students to the nature of scientific models, we adapted a brief 3-day unit on scientific models from Chinn, Duncan and Rinehart (in press). They then spent nine class periods on the first unit of designing an aquarium. In small groups, students drew paper models of what they thought needed to be in an aquarium. Through whole class discussion, these were combined into a class consensus list. Students next tackled the sub-problem of how many fish would fit into the 10 gallon aquarium in their classroom as they created paper models and did a brief gallery walk to critique each other’s models. The next day, they were introduced to CMP as a way to talk about systems and the Ecosystem modeling toolkit (EMT) as a tool for creating models. Building on their paper models, they used EMT to create aquarium models which they edited over the next class period as the teacher helped clarify how to use the software. As students edited their EMT models, the teacher orchestrated discussions about what the arrows meant, what properties of components were, and what it meant for sources of evidence to be credible. Following this, the students examined carrying capacity through a NetLogo simulation. Students continued this pattern of working in small groups with a mix of whole class and small group discussions. Worksheets helped scaffold their simulation investigations by asking students to make conjectures about relationships in the simulations, test hypotheses about these relationships, and interpret the results (see example in Appendix). As the evidence accumulated, students engaged in additional opportunities to revise their models. The students completed the aquarium unit with a gallery walk to review and comment on other students’ models.

The next unit was introduced with a video setting up the context of a fish kill in a local pond (classroom session 12 and lasting a total of 13 sessions). This unit was organized around the driving question of why the fish died. After jotting down individual responses, the teacher reviewed CMP and asked students to label their responses in terms of CMP and create initial EMT models. Over the next session, the class then reviewed the responses and sorted them into three categories: temperature, pollution, and something was missing (i.e., food, oxygen). Handouts provided evidence that students could use to support or refute models of these three possible causes. Students were able to determine which of several forms of evidence were consistent with two of the models and based on this revise their EMT models again, considering the evidence obtained thus far and which models they could rule out. The remainder of the class sessions consisted of working with micro- and macro-level pond simulations, exploring a CMP pond hypermedia, and a benchmark lesson on photosynthesis and cellular respiration. Benchmark lessons are employed to help learners deal with difficult concepts and to model inquiry practices in context (Singer et al. 2000) The simulations allowed learners to explore the factors related to decreased dissolved oxygen in the pond as they simulated the relationships between carbon and oxygen in a eutrophic pond. The macro-simulation allowed students to determine that as the algae die in large numbers, the dissolved oxygen decreases, killing the fish. The micro-simulation allowed them to peek into the microscopic processes that underlie what they observed in the macroscopic level simulation. In particular, they were able to observe that adding plant nutrients increased the growth of algae, and if excessive, the algae died off and decomposed, causing a rapid decrease in dissolved oxygen. When nutrient levels were not excessive, decomposing bacteria and algae fluctuated in a dynamic equilibrium. After manipulating the simulations and a mix of whole class and small group discussions, students revised their EMT models.

The last unit focused on marine phenomena. Students were introduced to three different phenomena that some scientists think may have a common cause: (a) dying oysters in eastern United States, (b) the shrinking of the Coral Reefs in the Florida Keys, and (c) disoriented Damselfish in the Florida Keys. The teacher split the class into 4–6 groups and 1–2 groups focused on each phenomenon. Each group was given data that allowed them to discover that their problem is caused by an increase in carbon dioxide level causing an increase in the acidity of the water. Over 10 class periods, using a similar cycle of modeling, engaging with simulations and hypermedia, the groups investigated the three marine phenomena. Each group presented their findings to the class and concluded that these three diverse problems shared a common cause. The driving question for the remainder of the unit; was “What can we do to store the carbon dioxide in the air long term, so that it will not increase the acidity of the ocean?” This established a need to know the details of the carbon cycle and conservation of matter. In order to explore this question, students used a Carbon Hypermedia and NetLogo simulation along with creating and revising EMT models.

Data source, coding, and analysis

In addition to their computer models, students completed a pre- and posttest focusing on systems-based relational thinking. The pre- and posttest were identical and consisted of a drawing task for both aquatic ecosystem and rainforest ecosystems (with a prompt asking students to draw and label relationships, several definitions in which students were asked to explain how terms were connected to aquatic ecosystems (e.g., animals, plants, oxygen), and additional questions designed to assess students’ systems understanding in ecosystem contexts (e.g., asking students to explain where and how cellular respiration occurs in different parts of an ecosystem).

To examine learning outcomes in the two classrooms, we coded pretests and posttests using a coding scheme derived from (Hmelo et al. 2000) and Hmelo-Silver et al. (2007) based on CMP representation modified from the SBF conceptual representation. The coding scheme is provided in Table 2. Because CMP coding assessed multiple system levels and their dynamic relationships, this was used as an indicator of systems thinking.

Table 2 Coding Scheme for CMP and M:M
Full size table

Components refer to the parts of the system. Any indications of components such as rocks, fish, plants, etc. were coded components. Mechanisms refer to underlying ecosystem processes. For example photosynthesis or decomposition are mechanisms. Any reference to processes was coded as mechanisms. Here we are defining Phenomena consistent with our earlier definition of function as the roles of components within systems or system outputs. For example, an illustration of algae producing oxygen would be an example of phenomena because it is the output pattern of the system. Mechanisms serve as the mediators between phenomena and components. Any mention of system elements relating to outputs or roles of components was coded as a phenomenon. We then looked at the extent to which these ideas were related to each other as shown in Table 2 (CMP). In addition, we coded the extent to which students identified macro- and micro-elements of the system (M:M). These coding schemes have been validated through expert review and the literature on systems learning. All test responses were coded blind for CMP and macro–micro by one rater. An independent rater coded a random sample of 20 % of the data. From this, we found inter-coder reliability to be greater than 90 %.

To analyze our data, we conducted a regression analysis to examine the effects of the treatment while factoring out pre-treatment differences for both M:M and CMP measures. We conducted additional exploratory analysis of the student drawings at pre- and posttests to examine qualitative differences over time and across treatment and comparison classes beginning with an examination of frequency distributions of the codes.

To illustrate how students in the treatment and comparison group differed, we examined the work of four students, two from the intervention and two from the comparison group in the drawing task that was part of the pre- and posttest. These examples are shown in Fig. 4. The student drawings were randomly selected and, for the intervention groups, electronic versions of the final group model were examined. With CMP and systems thinking as a lens, the drawings were first compared across classrooms, looking for similarities and difference. Next, the changes over time were compared for the individual students and between the two students in each treatment condition to identify qualitative changes that differed across the conditions. Finally, for the treatment condition, we looked at how the individual posttest drawing was related (or not) to the electronic EMT model that the groups constructed.

Fig. 4
figure 4

Example pre and post drawings

Full size image

Results

Overall learning gains

This small quasi-experimental study with a comparison classroom demonstrated significant learning gains for the treatment group relative to the comparison (see Table 3). Both regression analyses showed a statistically significant treatment effect: for M:M, Beta = 0.55, SE = 1.51 p < 0.001 and for CMP, Beta = 0.40, p < 0.001, SE = 4.21, p < 0.001. The effect size (d) was 1.35 for M:M, a large effect, and 0.87 for CMP, a moderate to large effect.

Table 3 Descriptive statistics: Means and Standard Deviations
Full size table

Qualitative evidence of changes in systems thinking

An important source of evidence of student understanding is the drawings that the students construct (e.g., Eberbach et al. 2012). Tables 4 and 5 show the frequency distributions of the CMP and Macro–Micro codes across classrooms for the drawing task that was the first item on the pre- and posttest. At pretest, the two classrooms show similar patterns. For the CMP coding, the modal response is a level 3, suggesting that students are connecting components to either a mechanism or phenomenon, with substantial numbers of student drawings being coded at lower levels. This pattern is further illuminated by the pattern in the macro–micro codes. The dominant pretest pattern is a score of 1—which always referred to a drawing that was at macro-scale. These cross-tabs also demonstrate that the intervention group improved in their CMP level, with 63 % of students receiving the maximum score with jumps of from 1 to 3 levels for 50 of 65 students. On the macro–micro dimension, 55 of 65 intervention students received the highest score, with most moving from the lowest level to the highest, indicating that they observed relationships among the macro- and micro-levels of an aquatic system. In contrast, the comparison students did not achieve the maximum score for CMP and did not show improvement on the macro–micro dimension.

Table 4 Student drawings: Pre- and posttest cross tabulations of CMP across classrooms
Full size table
Table 5 Student drawings: Pre- and posttest cross tabulations of Macro–Micro across classrooms
Full size table

To get a better sense of what these drawings look like, we examine some example drawings themselves, shown in Fig. 4. The similarity in all the pretest drawings is striking. In response to the prompt “Draw what happens in an aquatic ecosystem…” All the students in this sample have drawn a linear predator–prey marine system that largely includes living things other than water. At posttest, the qualitative differences between the students from the treatment and comparison are apparent. Student C, from the comparison class has elaborated the predator–prey story to a more expanded food chain that includes fish, plants, and micro-organisms and a side note on symbiotic relationships. There is an indication that the kelp uses the rays of the sun, but the mechanism is unspecified. Similarly, comparison student D also has a more elaborated food chain represented and includes some micro-level organisms. This student notes that algae and water are important for life, but provides no explanation of why this is the case. In contrast, two students from the intervention group make reference to key mechanisms. Both students A and B make reference to photosynthesis. Student A is vague but makes reference to some relation to oxygen through the line drawn from the link to plants and fish needing oxygen. Student A is focused more on the limiting factors for an ecosystem than details on the mechanisms. Limiting factors refer to those environmental conditions that limit the size of a population (e.g., plants need light, fish need oxygen). Intervention student B represented photosynthesis and decomposition explicitly in the drawing, noting that the light affects the algae such that they provide oxygen for the fish and use carbon dioxide; and noting that the fish decompose and that part of that process includes oxygen.

The two student examples from the intervention groups demonstrate that even though the diagrams suggest engagement with ecosystem processes by both students, student B has more elaborate descriptions of mechanism and an overall more connected drawing than student A. These two students were in different groups, and in examining the final group models (created electronically), we see indications that there was differential use of disciplinary content and connectedness in the group models as well. In Student A’s group, shown in Fig. 5a, there is some definition of decomposition and discussion of some of the elements, but there are also many rote facts provided. In contrast, Student B’s group, shown in Fig. 5b has more mechanistic explanations, such as the decomposing bacteria using up oxygen and nitrate supporting algae in photosynthesis. The explanations in Group B are more coherent and connected, though not yet wholly accurate or complete.

Fig. 5
figure 5

Example Models from a) Student A’s group b) Student B’s group

Full size image

Discussion

Findings from the study are timely as middle schools prepare to integrate NGSS in their curriculum. Science teachers stand to gain from incorporating key aspects of the curriculum design, instructional pedagogies and outcomes from this study to guide their curricular writing. Furthermore, this study serves as a model curriculum for middle school Science given that it addresses crosscutting concepts such as cause and effect: mechanism and explanation, systems and system models along with structure and function.

In summary, students in the intervention group deepened their understanding of ecosystem dynamics when compared to students who engaged in traditional instruction. In particular, students were better able to identify visible structures and to more completely account for causal mechanisms with relation to the phenomena. Although students still demonstrate some errors in disciplinary content, they are demonstrating important aspects of systems thinking. We contend that the combination of a conceptual representation and modeling practices has been shown to increase students’ understanding of natural systems. Our study joins others in supporting that the use of conceptual representation with models helps students to deepen their understanding of systems (Hmelo-Silver et al. 2011, Danish 2014); and that they are able to extend their ecosystem learning beyond a particular context through use of a conceptual representation across multiple ecosystems (Hmelo-Silver et al. in press).

Our data suggest that the intervention was a successful aid in the development of systems thinking among students in the study. Based on a previous study during the development of the intervention, we suggest the CMP embedded intervention encourages students to organize conceptual information using links between causal drivers (i.e., mechanisms) and outcomes/phenomena. There is converging evidence across the quantitative results and qualitative examples presented here that the simulations served as means for students to test ideas while the conceptual maps helped students to articulate and refine their understanding. Although further research is needed, we conjecture that the CMP framework may lead to more coherent systems thinking preceding a trajectory of more accurate disciplinary knowledge. This is consistent with results by Hmelo (1998) study of medical students in a problem-based learning curriculum, whose explanations became more coherent over time with accuracy coming later.

The structural elements of our intervention were nested in familiar contexts (e.g., aquaria, ponds, etc.) with supporting hypermedia. This helped students to draw on previous cognitive tools to explore new ideas. Recall in our intervention, students were provided with data that served to both support and refute their contentions along with critical questions that encouraged reflection and metacognition. This opportunity likely encouraged deeper thought into how parts of a system result in data-based outcomes.

The CMP representation also likely provided students with a means to organize and investigate the mechanistic elements and the parts/components of a system. In having students negotiate their ideas with CMP, they are reasoning about system elements in a more generic way. This process of abstraction allows students to relearn system ideas in novel contexts. Consistent with Preparation for Future Learning (Bransford and Schwartz, 1999) as a lens for learning transfer, students are likely using the CMP frame as a cognitive support to be used in other learning events (Sinha et al. 2013).

A limitation of our study was the quasi-experimental design. Students were not randomly assigned to conditions. In addition, our measures only focused on the content and not on the science practices (i.e., modeling) that were targeted. Future randomized comparisons will be important to address these limitations. Another limitation is that the study design does not allow us to separate the effect of the conceptual representation from engagement in modeling practices. Finally, we acknowledge that some of the use of mechanism may represent only the beginnings of systems thinking and the limitations of these paper and pencil assessments. For example, as part of the larger study, (Sinha et al. 2013) used interview data to demonstrate advances in mechanistic reasoning.

Our qualitative analysis suggests that individual student drawings and collaborative group models provide an indicator of student thinking and are promising for use in formative assessment. Moving forward, it will be important to use what we have learned to develop guideposts for teachers so they can use these student-constructed representations to adapt their support and guidance for teaching about systems thinking. Further analysis of student interactions will be helpful in identifying indicators of successful learning trajectories.

Implications for practice

Future directions include investigating students’ development of content knowledge and how they deal with data variability and modeling practices in relation to systems thinking content. While students are able to reason more completely after the intervention, we do not yet have evidence to support that students become more accurate with experience. Additionally, we have not developed material to encourage students to test variation in system rates and size. The latter would help students to test system outcomes under varying environmental conditions; a critical piece to understanding contemporary issues such as climate change. Finally, we need to consider how CMP might be generalized as a tool for teaching systems thinking more broadly. We reiterate Sabelli’s (2006) contention that systems thinking is an imperative for scientifically literate citizens who can reason about the pressing problems facing the contemporary world.