9.1 Introduction

Epistemology in the biological sciences includes approaches that respect the connectedness within systems, the value of accumulated evidence, long time scales and attention to steady states in addition to change. Helping students to understand the epistemic origins of biological knowledge—how we come to understand and agree upon knowledge based upon how evidence is gathered and claims are made in the discipline—is critical to their deep understanding and appreciation of it. Yet often, science education focuses on disciplinary knowledge to the exclusion of how knowledge is generated, how epistemology differs across scientific disciplines, and how such assumptions relate to the inherent complexity and connectedness of systems concepts. When there is a focus on scientific practices, these seldom reach down to the level of the epistemic assumptions underlying the generation of knowledge in the biological sciences. In this chapter, we focus specifically on ecosystems science as an exemplar within the biological sciences.

Efforts to introduce students to epistemology in science are often narrowly framed as an isolation and control of variables approach and a stereotyped version of the scientific method; centered on lab-based approaches and manipulatable phenomena, these efforts ignore approaches that draw upon accumulated evidence (Sinatra & Hofer, 2016) as in the biological sciences. A focus on isolating and controlling for variables misses the larger complex causal dynamics in ecosystems and does not begin to approach how ecosystems scientists engage in research. Helping students to learn the multitude of ways that ecosystems scientists develop evidence-based explanations for systems concepts in a complex world should be an essential part of biological education.

Yet even as science education reform documents push beyond a single, narrow version of the scientific method, resources for supporting teachers’ efforts to broaden the epistemological approaches that students are exposed to are relatively few (Kamarainen & Grotzer, 2019). A study of how K-12 teachers understand epistemologically authentic approaches in ecosystems found that while they held relatively sophisticated perspectives on the diverse approaches used by ecosystem scientists, these were not reflected in the descriptions of their practices within their ecosystem science units. The results suggest the need to support teachers in adopting the epistemologically authentic practices within ecosystems science and in finding ways to translate those to the classroom (Kamarainen et al., 2021).

This chapter argues for the importance of introducing students to the epistemic assumptions that biological scientists make when framing their work particularly as they relate to understanding the connectedness of systems. It discusses an approach called a “Body of Evidence Approach” (BOE) for analyzing the causal complexity of ecosystems and introduces a study that was conducted to assess the impact of teaching this approach to middle school students.

9.1.1 What Is a Body of Evidence Approach?

Through a series of open-ended interviews with ecosystems scientists, Kamarainen and Grotzer (2019) found that they characterize causal patterns and relationships in ecosystems as embedded in a complex matrix of interactions, subject to inherent and sometimes irreducible variability and not always subjectable to manipulative experimentation. In these situations, they use strategies to construct understanding of complex systems through constructing a Body of Evidence approach—integrating results of multiple approaches, measuring and describing variability, conducting experiments in context, taking advantage of natural experiments, thinking across levels and considering the limits to generalizability. Ecosystems scientists also demonstrated considerable “epistemic fluency”—referring to the ability to discern and engage in a variety of investigative approaches for gaining knowledge in a certain field (Goodyear & Zenios, 2007).

At the core of a BOE approach are ways to think about the nature of causality that fit with the information available to ecosystems scientists. Kamarainen and Grotzer (2019) argue that, “Moving from a correlational to a causal account involves epistemological assumptions in any discipline. It presents particular challenges when phenomena involve multiple causes, time-lags, feedbacks, or thresholds as is the case in ecosystem science. While reductionist approaches may contribute to explanatory efforts, investigation in ecosystems science requires a systems perspective” (p. 533). The ecosystems scientists in the study pushed against the notion that complex systems can be understood through reduction alone. They argued for the use of confirmatory approaches such as developing complementary possible models to consider, holding models in consideration until enough evidence exists to determine that they are clearly wrong, considering multiple lines of evidence and the soundness of the evidence for possible mechanisms (Pickett et al., 1994). They also argued for using ‘natural experiments’ and non-traditional forms of experimentation (Jensen et al., 2012) for instance, dividing a pond with a nylon curtain and treating one side and not the other as a comparison study (Bennett & Schipanski (2013) or placing a tent over a steam to exclude leaf litter and accessing the impact over time (Strayer, 2013).

9.1.2 A BOE Approach for Middle School Science: Understanding Goals

How might this approach translate to what students in secondary school are taught? The following set of Understanding Goals represents the substance of how we approached teaching students about the ways that ecosystems scientists think about causality in complex contexts:

  1. 1.

    It is not always possible or desirable to conduct an experiment.

  2. 2.

    When it is not possible or desirable to do an experiment, ecosystem scientists use an approach where they systematically look for lots of different types of evidence. (They call this a “Body of Evidence” approach.)

  3. 3.

    The more evidence that can be gathered in support of a claim, the more likely it is that that the claim will be accepted. The evidence should be from different and varied sources.

  4. 4.

    In addition to trying to find out what makes something happen, scientists try to collect as much information as they can on how the cause and effect relationship varies—the range of possible outcomes. (For example, a variable might cause an outcome when it reaches a certain amount, but not at lesser amounts. It also might not cause more of an outcome as you keep adding more. Or it might be that the amount of outcome increases stepwise with the amount of the causal variable.)

  5. 5.

    Sometimes nature “conducts experiments” that scientists can interpret. They use these as natural opportunities to learn about what happens. Natural opportunities can be especially helpful in cases when an experiment is not possible or desirable.

  6. 6.

    Scientists talk about how much certainty they have in a set of findings. They may express uncertainty. They may express certainty at different levels of analysis of a problem and not at others. They may talk about certainty in some contexts but say that it is not generalizable to other contexts (limits to generalizability).

9.2 Research Questions

The case study described below incorporated these Understanding Goals (UGs) into a broader ecosystems science unit to investigate how students responded to the BOE UGs and how it influenced their learning. Their understanding was contrasted to students who experienced the unit without the BOE components. The study sought to address the following questions:

  1. 1.

    What characterizes the understanding of students in each class?

  2. 2.

    What can be learned from the contrast about helping middle school students to learn about BOE as an approach in ecosystems science that can inform future educational efforts?

We hypothesized that students experiencing the BOE components would demonstrate understandings that are more closely aligned with the UGs above and that they would be more likely to seek out multiple and corroborating forms of evidence.

9.3 Methods

9.3.1 Design

The impact of teaching a BOE approach to middle school students learning about ecosystems complexity was explored by comparing the understanding of students in two classes from an urban school who participated in a Problem-Based Learning (PBL) Curriculum called EcoXPT (with the XPT used as shorthand to capture its specific focus on experimentation within ecosystems science) in a case study. Both classes participated in the curriculum; One class had additional instructional components related to a BOE approach infused (EcoXPT+ BOE) while the other did not (EcoXPT). Assessments included concept-maps (containing evidence for each claim) and post-interviews. Students also took an on-line pre- and post-inventory as in other studies of the impact of the EcoXPT curriculum, but given the small n, no clear patterns emerged so this data is not included here. The pre-inventory data suggested that the groups were equal upon expectation at the outset of the study.

9.3.2 Participants

Two seventh grade classes of the same teacher participated. The students were from an inner-city charter school with 91% minority enrollment, low socio-economic status (SES) and low scores on a state-wide, standardized test (the Massachusetts Comprehensive Assessment System (MCAS)) (Math Proficiency—27%; Reading Proficiency—34%). A total of 22 students participated with 12 students in EcoXPT and 10 students in EcoXPT + BOE. Students worked on the curriculum in pairs and participated in both individualized and pair assessments. As a case study, the sample size was quite small and thus is a limitation to the generalizability of the findings.

9.3.3 Curriculum

Both classes participated in the EcoXPT curriculum for 14 days (See overview in Appendix). The curriculum centers on an immersive virtual world that depicts a pond ecosystem. (See Fig. 9.1.) (A full description of the EcoXPT curriculum and the Teacher’s Guide are available here: https://ecolearn.gse.harvard.edu/projects/ecoxpt). Students worked in teams of two to a computer. During Curriculum Days 1–3, they explored the world and were instructed to “get to know it.” On about the third Curriculum Day, students discovered that a fish die-off had occurred on a certain day within the virtual world. They then began to investigate possible causes for the fish die-off by traveling in virtual time before and after the event to observe and collect data on population levels and water quality measurements. They used data tools in the world to view and graph this data which allowed them to see patterns between the different types of data. On Curriculum Day 7, students in both classes had access to experimental tools and to scientists in the world who shared the rationale for certain kinds of approaches and their epistemological assumptions. The experimental tools varied widely to include lab-based experimental tools such as tolerance tanks and comparison tanks and in-situ experimental tools such as mesocosms, tracers and a water buoy that collected data over a period of virtual months. (See in-depth descriptions of these tools in Appendix.) Other sources of information available to students included observation of organisms, changes over time, a microscope that allowed them to zoom in on microscopic organisms in the pond, the testimony of virtual scientists and other non-player characters in the world and a field guide that offered information about the organisms that they found in the ecosystem.

Fig. 9.1
figure 1

Image of the EcoXPT immersive world

Full size image

Throughout the curriculum, students in both classes were introduced to a set of science-related thinking moves, illustrated by classroom posters that they could refer back to (thus reducing the memory demands of learning the thinking moves) while working in EcoXPT and short explanatory videos supported by in-depth teacher support notes to help students to learn how ecosystems scientists engage in scientific practices. These included a set of five posters for both groups: Deep Seeing which asks students to engage in careful observation of the ecosystem; Evidence Seeking which stresses the importance of using evidence to support one’s claims; Pattern Seeking which asks student to look for patterns in the on-going processes and steady states of the system; Analyzing Causality which encourages students to seek out the causal mechanisms relevant to the relationship; and Constructing Explanations which asks students to put together the available evidence to construct the best causal story or explanation that they can. (See Appendix.)

9.3.4 BOE Intervention Components

It is possible to add elements into the virtual world and to turn their appearance on or off. Using this capability, additions were made to the virtual world in the BOE condition. So, while both classes worked in a rich, contextualized immersive simulation, certain elements were present only for those in the BOE class. These included: additional dialogue by some of the virtual scientists in the world. For instance, students in both groups find Dr. Jabir standing outside near a set of mesocosm experiments. In both groups, he says, “Let me tell you about the mesocosm experiment I’ve been running here for the last two weeks.” However, for the BOE group, first he says, “Lab experiments are great for isolating and controlling variables, such as whether phosphate level affects fish. But mesocosms let us consider how other variables in the real world, like sunlight and temperature, interact with variables we are testing. Even though it is not the same as experimenting on the actual pond, it gives us more certainty about what is going on without hurting the pond.” The BOE students also meet Dr. Aziza Al Dahan standing near Amelia Pond which has turned bright green. She says, “Hello, are you noticing what I am noticing? This small pond has turned bright green. What do you think is going on? Think about it for a little while and then come back to talk to me.” Later, they run into her again and she says, “I investigated and found out that a farm worker put a new manure pile in a place where the run-off comes to this pond. This has caused a spike in the algae levels in the pond.” “As an ecosystems scientist, there are times when something happens that was not intended, but that we can learn from. We wouldn’t have done an experiment directly on this little pond, but now that it has happened, I am studying it to learn from what happened.” “In this case, a person made this happen, but sometimes nature creates opportunities to study changes in the ecosystem—such as when a hurricane comes through and changes a landscape or fire removes the smaller trees and understory from a forest.” “Does anything that you are seeing here corroborate what you are finding out about Scheele Pond?”

Students in the BOE class were introduced to the BOE Thinking Move and saw a poster and accompanying video (See Appendix for script) explaining the Body of Evidence Approach, encouraging them to collect evidence from multiple sources, look for corroborating evidence of different types (including perceptual evidence, patterns in data and graphs, numerical information and testimony from trusted others) and to assess the validity and reliability of the sources of evidence. They used a Body of Evidence Worksheet (See Appendix) to consider how more evidence leads to greater certainty/stronger claims and worked with a partner to evaluate evidence through a BOE lens. Both classes did talk about different types of evidence and what can be learned from different types as part of the Evidence-Seeking Move and filled out a worksheet focused on this topic (See Appendix), however only the Plus BOE group discussed it within a BOE framework. Supporting materials included ways to talk about BOE for both students and teachers (see Appendix).

9.4 Data Sources and Analysis

9.4.1 Concept Maps

Each pair of students developed an online concept map of their understanding of the causal dynamics within the virtual ecosystem. On the sixth day of the curriculum, students in both classes were introduced to a concept-mapping tool in the immersive world. (See Figs. 9.2 and 9.3.) Students choose from a set of images of factors that become the nodes of the map. They then define claims by drawing an arrow from one node to another and by choosing how to label the arrow (affects, does not affect) (for example, “phosphates affect green algae”). A button on the claim window invites them to state the evidence for the connection. This button opens to an index from the on-line notebook within the program where they track things that they are finding out in the virtual world. It offers evidence that is linked to the source that students used to identify the evidence. A text box asks students to explain their reasoning for how the evidence bears on the claim. For the purposes of this study, the concept mapping tool was modified to include a number along each arrow to alert the students as to how many pieces of evidence they had used to explain each arrow in their concept map. (See Fig. 9.4.) A box appears on the lines of each connection in the concept map to show how many pieces of evidence students provide for that given connection.

Fig. 9.2
figure 2

Concept-mapping tool

Full size image
Fig. 9.3
figure 3

Links to evidence within the tools

Full size image
Fig. 9.4
figure 4

Boxes along lines in concept map show number of pieces of evidence

Full size image

An analysis of student concept maps was conducted to look for potential differences in the number of connections, amount of evidence provided for each connection, as well as the percentage of connections for which they provided both evidence and reasoning. The maps were coded for the different types of evidence provided in each connection for each pair’s concept map and the frequency of use of different types of evidence was calculated to offer a quantitative view of how they considered evidence. These frequencies were calculated into class averages. A second round of coding analyzed the concept maps qualitatively. Major themes were identified from the connections that students made, the factors that they used, the evidence they provided in relation to the connections they made, as well as the type of causal connection they made (in that, “does not affect,” “affects each other,” “affects”).

9.4.2 Post-interviews

A set of interview questions focused on epistemology in ecosystems science and BOE were developed. It included questions such as: “What are some things that ecosystems scientists do when they cannot conduct an experiment and want to make causal claims about what has happened in the ecosystem?” “Have you ever heard the term “Body of Evidence”? If yes, what do you think it means? If no, a Body of Evidence means “a collection of evidence.” What do you think that means?” Why is it important for ecosystems scientists to collect a “Body of Evidence” or a “Collection of Evidence”? What are some reasons that they do it?” These questions were followed up with open-ended probes, such as, Can you tell me more? What does that word mean to you? etc., to get at students’ intended meanings. Following the intervention, eight students, four from each class, were interviewed in sessions that lasted approximately 30 min. This analysis included an emic process of surfacing emergent themes within each interview (Strauss & Corbin, 1967), capturing these as a memo about each subject’s knowledge and then looking across memos for themes across interviews generally and as clustered by the two classes.

9.5 Results

9.5.1 Concept Maps

As seen in Table 9.1, EcoXPT students made more connections but had fewer pieces of evidence within their connections than students in the EcoXPT plus BOE students. They also did not include both evidence and reasoning in their connections as much as EcoXPT plus BOE group. The EcoXPT plus BOE students made fewer connections but had more pieces of evidence within their connections. They also had higher rates of both evidence and reasoning in their connections. The EcoXPT Only students tried to use more factors to make more connections and to construct more of the causal story. This response pattern was frequently witnessed with students using less relevant reasoning to the connection they were constructing, but it would often help tell the “story” (in that reasoning often explained the connection or elaborated on it by “telling parts of the story.”). Some students also didn’t make broader or more complex connections, but instead they made minor connections (only two or three factors) across the causal scenario. They also used the same piece of evidence for multiple connections even if it wasn’t the best evidence for a given connection. The EcoXPT with BOE students generally made fewer connections and there was a tighter range with less variability in the number of connections that they made. However, they included more of the “foundational” part of the story (usually related to fish, herons, abiotic factors you can test in a lab, etc.) and they tried supporting it well with evidence and reasoning. The tighter range with less variability may be due to the greater cognitive load of processing more evidence for each piece. It is not possible to know conclusively given the small n in each class and might also represent differences in the student samples.

Table 9.1 Concept map comparisons in EcoXPT vs. EcoXPT + BOE classes
Full size table

The most frequently used types of evidence across both groups were the tolerance tank results and the field guide. (See Figs. 9.5 and 9.6.) This makes sense because of the progression of the Thinking Moves and curriculum. The curriculum encourages observing and taking notes (or looking at notes, as in the field guide) and then analyzing causality (which is usually first explored through the use of the tolerance tanks in the lab because it’s easy to isolate direct causal relationships). The BOE class tended to refer to pattern data (in the graphs) to a greater extent whereas the EcoXPT Only class used testimony to a greater extent.

Fig. 9.5
figure 5

Evidence use by EcoXPT only students

Full size image
Fig. 9.6
figure 6

Evidence use by EcoXPT Plus BOE students

Full size image

9.5.2 Interviews

The interviews suggest that some students in the BOE groups were able to grasp some aspects of a BOE perspective including the importance of varied sources, the belief that ecosystems scientists try to study the environment in ways that don’t harm it—a “do-no-harm” perspective—and the value of holding multiple possible models in consideration, as elaborated below. Their ability to reflect explicitly on the framing for their evidence and the encompassing epistemology was somewhat more limited than anticipated and there were some clear challenges as well. The following themes were evident.

9.5.2.1 Confounding Causal Factors with Sources of Evidence

Students focused more on multiple causal factors involved in the eutrophication scenario than on the meta-level concept of multiple sources of evidence as necessary to support causal claims. When questioned about sources, they tended to confound them with factors. This response pattern occurred to some extent in both groups.

Most students gave sophisticated explanations of the complex causal factors involved in the eutrophication scenario. They seemed well able to think about complexity and multicausal scenarios and explained how causality might work. However, when asked about sources, students in both groups conflated multi-causal explanatory factors with the concept of multiple sources of evidence (as exemplified by subjects 3, 4 and 6 below).

Some students focused this part of the conversation on what they needed to do to find evidence—the behaviors that they would engage in, to a greater extent than the nature of the evidence itself and the meta-concept of the epistemology and a strong body of evidence. For instance:

  • Subject 3 (EcoXPT Only):

    • I-…can you give me examples, more examples, when you say they are collecting evidence?

    • S3-They go to, they test the water, to see if the water is okay for animals to live. They check the soil and the plants to see if okay, that plant life can breathe. They try to find animals, see if there’s a lot of animals or not that many animals.

    • I-…Yeah, you’re kind of talking about this claim, there’s evidence. Tell me more about how they connect.

    • S3-Oh they connect because we looked at the days- the day before they died, the day after they died and the day they died. And we looked for a pattern, something changed. And we thought the dissolved oxygen went down a little bit and [inaudible]

    • I-So in that case you’re saying the claim-...

    • S3-Was like the dissolved oxygen killed the fish- was the cause of the fish died.

    • I-And what was the evidence?

    • S3-The evidence was that the day before the fish died, I think like the dissolved oxygen was perfect and then when the fish died the dissolved oxygen went I think either down or up.

    • I-OK. …why is it important for ecosystem scientists to collect a body of evidence, or a collection of evidence? And what are some of the reasons they might do it?

    • S3-It’s important for them to collect because they want to say how the ecosystem changed in that time they collect evidence. And it’s important because what if something bad happened to the ecosystem and they didn’t collect evidence they don’t know why, or what caused it.

  • Subject 4 (EcoXPT Only):

    • I-Tell me more about that, what do you mean by evidence to support their reasoning about the ecosystem?

    • S4-Well, evidence and their reasoning, they’re going to want to find more evidence that actually is relevant for what they think is the reason why the ecosystem is damaged, or succeeding, or whatever state it’s in.

    • I-Can you think of some examples about that type of evidence, or what they might be doing, ecosystem scientists?

    • S4-Well they would do, probably use tracers, so harmless chemicals that give off a glow, put them into an ecosystem to see what kinds of factors are going inside the water, or doing these certain things and affecting these. …Some other evidence is like, trying to find out populations. Trying to find out about the microorganisms, because microorganisms are a very big part of ecosystems, a very big part. …Because microorganisms, you may not see them, but they do a lot of things. Bacteria can travel real far. So you also gotta measure bacteria the most. There’s a type of bacteria that’s from Japan, that’s native to Japan, that was found on a person, he lived in the USA. But the thing was, this person had never been to Japan. That is how far bacteria can travel. It can travel in and out of ecosystems, just like that.

    • I-So do you think that’s something ecosystem scientists are doing?

    • S4-They’re trying to see mostly all the factors. Mostly all, not really all because it’s hard to find all factors. If you don’t have all the stuff you try to find most of them, so that makes the most sense.

  • Subject 6 (Plus BOE):

    • S6-Ok. Umm… Um, algae, bacteria levels. So algae multiplied because of the heat and the sunlight. They started growing too much. And then after it grew too much they died and then all the plants below start dying too because of lack of sunlight. Then bacteria started decomposing it and then the bacteria levels started increasing which made the dissolved oxygen levels decrease. So that’s when it’s being caused. Algae caused bacteria to increase which caused dissolved oxygen to decrease.

    • …Well, they can get data and turn it into a graph to figure out how and what is causing the other one. Like the example that I used before- algae, bacteria and DO levels. That’s the one that’s causing it. And it could also be like something I thought before about the herons eating all the fish, so that was my answer at first because the heron population was increasing and the fish population was decreasing. So then that was my main, like source. But then I started thinking about it and then I figured out that wasn’t correct because if the food source is going down then so should the predators.

This focus on figuring out the factors involved in explaining what happened at the pond and on the actions that one would engage in to find out, by subjects in both conditions, is not, in retrospect, surprising. The response has more behavioral coherence with what they are being asked to do in EcoXPT—to investigate and develop an explanation for what has happened to the fish. Focusing on the sources of evidence introduces a meta-level to that process. It is possible that the increased cognitive load of the task was more than students in both groups were able to engage with. As considered in the discussion, this raises the question about whether there are instructionally more effective ways to get students to focus on sources of evidence—such as evaluating someone else’s data and deciding whether information is trustworthy.

Two students, subject 5 from the BOE class and subject 2 from the EcoXPT Only class, clearly did differentiate between factors and sources—not confounding them—and talked about the importance of multiple sources, suggesting that students are able to do so. Subject 5 (BOE) talks about the importance of multiple sources of information in providing for a body of evidence. For example:

  • S5-Because you can’t really rely on one source of something because you have to get a lot of sources to see if they match up. And some news can be fake from what you’ve heard, so you have to learn from other sources.

  • I-…do you think it’s better to have multiple different types of evidence or one type?

  • S5-Um the same thing, multiple types of evidence because the thinking move body of evidence has something to do because you can maybe mark out like one claim we had about it and then something debunks it, we can’t really see what debunks our claim of the situation if we only have one source of evidence.

  • I:-Could you tell me what you think the term body of evidence means?

  • S5-Um, it says body in it, so it means like, I think a lot of things like a lot of evidence to debunk your claim or support your claim, or maybe new questions along the way.

  • I-Is there anything else we should know about body of evidence?

  • S5-Um, don’t have like, don’t be disappointed or have high hopes. Just clear your mind and just not think about it too much. Or have an open mind about it.

  • I-Should they be different sources, same sources?

  • S5-I think they should be both so you can have a variety. …so you can have different types of sources to go back to. ….I guess it would be nice to have a variety in my opinion. I’d rather have a lot of things instead of like one source. I’d rather have a variety of sources.

  • I-Why is that important?

  • S5-Very reliable. It’s very reliable to have a lot of sources or a lot of things.

Subject 2, an EcoXPT Only class member, also differentiated between sources and factors. However, s/he insisted that evidence that is all from the same source is better, that if the evidence came from different sources that would not be as good. While s/he recognized that multiple pieces of evidence are helpful and gave a preference for experimental evidence, s/he explicitly rejects varied sources. It is possible that the student was expressing a distrust of testimony in particular and the need for additional evidence to back it up, but the interviewer did not probe this aspect of understanding further. For example:

  • I-Well we’re just talking about having multiple pieces and are all of the pieces of evidence from the same source, or different sources?

  • S2-All from the same source.

  • I-Let me think of an example. Pretend my claim is, dissolved oxygen affects fish and I have two pieces of evidence and they’re both testimony, so I heard someone say something. I heard a scientist say “I know from my experimental experience that low dissolved oxygen causes fish to die,” and someone at the pond said “I heard dissolved oxygen can cause the fish to die” Is it better to have two of the same type of evidence, does that make it strong?

  • S2-Yeah, because they both gave you really big pieces of evidence. And you could probably get something out of it, too, by yourself or something. And then probably make it stronger and stronger.

  • I-What if you had two different types of evidence? Let’s say you had testimony and you had data. Is that also good or not as good?

  • S2-I think it’s not as good because the testimony, you’re testing it- I’d try the experiment thing though, because what people- what if they don’t know what to do and they just said it to you? I would test it to see if it actually works. I would test it because it would be- you could get a lot of evidence out of it.

  • I-…Is it better to have two of the same- for example two people said something. Or is it better to have evidence from two different sources? So for example someone said something and data you collected.

  • S2-I think the same

  • I-because…?

  • S2-Because if they’re thinking the same way, they can probably, we can all work together, try to find something else.

9.5.2.2 Expressing the Value of Multiple Possible Explanations/Models

At least one student expressed the value of multiple possible explanations and holding different possible models in mind (which is different from multiple factors, multi-causal explanation), connecting it to open-mindedness.

Subject 8, a student in the BOE Class expressed that s/he was holding more than one possible explanation in mind implying an open-mindedness that fits with the epistemic value of holding multiple possible models and the uncertainty and humility with which the ecosystems scientists considered possible explanations and supporting evidence (Kamarainen & Grotzer, 2019).

  • S8-And if the algae probably doesn’t, this is my other theory- if the algae doesn’t uh, like if the algae doesn’t uh… Ok so, if the algae is like not working right, like if the algae is uh… Let’s see… Oh yeah, so if the algae like uh… If the algae uh… Can you skip it? Skip it. Because I can’t have an explanation for it.

  • I-Yeah, do you want to think about it for a second? We have time.

  • S8-Uh, yeah sure. Uh… Oh ok! So I got it now!

  • I-Yeah.

  • S8-So the nitrates probably activated the algae, so the algae can be produced more, the algae might cover up the, like the top of uh, the ceiling of water. And it probably can’t make the photosynthesis go to the water, so the fish can’t, so the plants down there can’t do photosynthesis so they can’t make oxygen for the fish. And when it does that, the fish is going to die because oxygen. And the bacteria is going to break it down and also there’s more bacteria, bacteria also takes up the water, it takes up more water, the fish might not have a lot of water to breathe. So I think something caused the algae, like nitrates, or nitrogen probably taking up the oxygen I think.

9.5.2.3 Recognizing a Collection of Evidence Intended to Support a Claim

Interviewees in the BOE Class offered descriptions of BOE that included recognition that the evidence is a collection, not just randomly chosen and that the intent is to prove a claim.

Subject 7 made explicit comments to indicate that a Body of Evidence has meaning beyond just a bunch of random evidence, for example:

  • S7-They collect a bunch of evidence for um, so they will have more than one reason of why something would cause the other or something that happened. …A body of evidence, um… So a body of evidence is this like, collecting evidence and running tests to see if your right or wrong every day until you get it right eventually, or if you fail. And a bunch of evidence is like, just having evidence and basically not doing anything with it. I guess.

S8 viewed BOE as needing to prove claims (though not necessarily in the context of broader epistemological considerations).

Subject 8 (BOE):

  • I-Have you ever heard the term body of evidence?

  • S8-Yes.

  • I-Ok, what does it mean? Can you tell me?

  • S8-It means like, say for scientists there’s like an experiment, you need evidence to show the experiment that he did or she did, so he or she would probably be doing an experiment about fish and they will see what happens in the ecosystem like five days after it rained. They need a body of evidence, so they need to take like notes, or like body work, or they need to like take pictures, to see how many, like a lot of body of evidence that they have to include and talk to other scientists to see what the can do to fix it.

  • I-Ah, ok. So tell me more. You mentioned they might have notes, they might have like temperature, things like that. Tell me what that means in terms of the body of evidence, like what is it?

  • S8-They’d probably have to take notes like, uh, like the turbidity went down below like 50 degrees, or uh, 50 Celsius I think. And they had to that and see what happens like everyday and see what happens during the day, see where body of evidence is- how much evidence that they have.

  • I-And so why, why is it important that ecosystems scientists collect a body of evidence? Like what are some reasons why they would do it?

  • S8-Because they need body of evidence, because if they just say it then like, oh yeah that happens, but then they need like a claim that can like prove their explanation.

  • I-So is this something that you did when you did EcoXPT? Did you think about this?

  • S8-Yeah I thought about that if I just say something I need to back up with evidence. If I just say that the algae killed the fish I have to go and research and have to go around and do temperature, water temperature, see what happens on the algae on this day, on the day that the fish died, the population of the algae, the green algae, blue algae and see what happens. Because if you say like it’s algae- prove it! So I have to go out, search for stuff, research it, get like a body of evidence. So, yeah…

9.5.2.4 Making Connections to Other Learning about Evidence

Despite the lack of introduction to BOE as a concept, when asked what it might mean, students in the EcoXPT Only class made connections to what they had learned about providing evidence when writing in general and in science, in particular.

Students in the EcoXPT Only group made connections to an idea that they appear to have learned in class about writing and perhaps scientific writing in general where you have an introduction, then a body of evidence, then a conclusion. For example:

  • Subject 1 (EcoXPT Only):

    • S1-So, body of evidence could be- like in an essay you have the introduction which has a claim. Then you have the body of the claim, which has three, two, or some, or a certain amount of body evidence in your essay. …It’s like an essay, when you have the body evidence, which is the evidence from the claim that you’re making.

  • Subject 3 (EcoXPT Only):

    • S3-Well I think like a hamburger, is like the meat is the middle, what you’re doing, like an essay- no, not like an essay. Body, a paragraph- the middle of the, experiment? If you have evidence… …like if I explain something, I’m going to have to have evidence. So I’m going to have like the middle.

    • I-So what does the term “body of evidence” or a collection of evidence in the context of ecosystems, what do you think that might mean? OR what we might use it for?

    • S3-To see if the evidence goes with the claim you are trying to make.

    • I-Okay, tell me more, why is that important?

    • S3-Because if the evidence has nothing to do with the claim you’re trying to make, it’s going to be hard to back up your claim.

9.5.2.5 Acknowledging Ecosystems Science Experimentation as Sensitive to Not Harming the Environment

Students in the BOE Class talked about how ecosystems scientists attempt to “do no harm” and how this impacts experiments that they would not do.

Students in the BOE Group did seem to come away with a clear sense that ecosystems scientists try to do “do no harm” investigation—that they don’t burn down the forest to see what happens and that they might investigate things that naturally occur (what is called “natural experiments” or “opportunistic investigation” in the curriculum) even if the students did not explicitly use those words. For example:

  • Subject 8 (BOE)

    • I-What are some things that ecosystem scientists do when they can’t conduct an experiment but they want to make a causal claim?

    • S8-Uhh, they probably might, uh let’s see… Maybe they might do a tolerance tank, they might get a lot of fish and put a lot of stuff inside to see what kills them because they can’t go outside and do that to all the fish to the ecosystem because they, it won’t be good.. So yeah, the fish would probably die really fast, so they had to do it in a tank and see what happens. Or they probably, um, they probably just do something that makes like, I’m not really sure, but I have an example. So say like, they want to go and see how many forest fires happened that one year, not forest fire that happened every once a year, a forest fire happens like, how high the temperature of the forest fire can get. …They’re not going to go burn a forest fire and burn it. So they might probably get like a couple of branches and trees and they might like put it in their office and burn on fire and see what happens.

  • Subject 5 (BOE):

    • S5-So, if something like in the past, a lot of things happen in the US here or worldwide, so um, like a forest fire. They can’t just put a forest fire on an actual forest because it could hurt the ecosystems in there that it’s producing. And they could use some research of it from recent years, or how long it is and see the effect of it how it was before.

    • I-Anything else?

    • S5-Um, an internet one? Like an online one, like how you did it with EcoXPT without actually harming any fish in real life. ….Oh wait, what if they get already passed on fish and conduct an experiment on that?

The interviews suggest evidence that some students are able to understand and use aspects of Body of Evidence reasoning, but also that the ability of the broader group of students to reflect explicitly on the framing for their evidence and the encompassing epistemology was somewhat more limited than anticipated.

9.6 Discussion

The findings suggest that there were subtle shifts in how students viewed the importance of evidence in support of their claims and that the BOE students focused on constructing a compelling body of evidence in support of each claim. Some of the interviews indicated an appreciation for holding different possible models in mind and considering multiple lines of evidence as the scientists did. It may have come at the expense of a fuller explanation of the complex causal scenario as they constructed less of the explanation in their concept maps. At the same time, their explanations focused on causal dynamics that were central to the eutrophication scenario. Given the period of time over which the curriculum plays out and the primary tasks of investigating the reasons behind the fish die-off, it makes sense that a focus on BOE in addition to constructing the causal connections would divide students’ time and attention to some extent.

Both groups of students revealed understanding of the importance of evidence. The BOE group used more evidence in support of their concept map connections. Some of the BOE interviewees were able to talk explicitly about constructing a set of evidence and the importance of having corroborating evidence. The EcoXPT Only students also thought carefully about the types of evidence and the importance of providing support for testimony. The EcoXPT curriculum materials support these understandings (even without BOE framing) through the Evidence-Seeking Thinking Move and the supporting materials (See Appendix).

The findings also suggest that students in both groups focused more on talking about developing explanations with multiple causal connections and including multiple possible facts than multiple sources of information/evidence for each factor. This response pattern makes sense given that developing the causal story is more directly aligned with the primary goal of figuring out what happened to the fish and the other is at the meta-level of how they establish the causal connections and generate the scientific knowledge behind their explanations. The finding that the BOE focus translated into a stronger focus on collecting corroborating evidence even if it did not result in as much of an explicit awareness of the epistemology as anticipated is a step towards acknowledging the epistemological underpinnings.

There was clear evidence that BOE students understood that ecosystems scientists needed to find ways to construct causal explanations that did not harm the environment, that they entertain different possible explanations, and that they focus on constructing strong explanations and that this relates to the evidence that they can provide to support their explanations. Students in both classes expressed understanding of the value of experimentation in providing evidence.

This initial, exploratory study is promising, but limited in what can be learned from it. One could imagine that a longitudinal study would reveal more about how these understandings can be developed over time. With greater time to build such understandings, it seems that it would be possible to build the meta-level understandings about varied and multiple evidence and constructing a powerful explanation as an integrated part of figuring out what one believes to be the causal connections. A longer and somewhat larger study might also reveal particular points of difficulty in learning these ideas—both where they are challenging to learn and where they may interact with other concepts in ways that could lead to misunderstandings. It is also possible that students would learn more about the particulars of how ecosystems scientists construct complex causal explanations of systems if they had opportunities to contrast these epistemologies to approaches in other areas of the sciences as such juxtaposition would help to highlight the features of each. Despite its limitations, this study does suggest possibility and promise for developing important understandings in how biological systems are understood.