The National Research Council’s Framework for K-12 Science Education (2012) has recommended a move away from memorization towards a focus on scientific and engineering practices that utilize technological modeling tools in order to promote deeper understanding of science concepts. The use and development of models in science education will be new to most educators, as there are not many curricula or professional development options that focus on model-based reasoning (Haag and Megowan 2015). The use of engineering practices in science education will also be a new hurdle for teachers to overcome. Inroads on these challenges have been made in high school physics (Jackson et al. 2008) and engineering (Zeid et al. 2014). However, biology curriculum focused on modeling and engineering has been overlooked, which is an unfortunate oversight since biology is experienced by far more high school students. We present an approach to modeling and engineering that can be broadly implemented in high school biology classrooms, focusing on a particularly challenging topic for investigation by high school students.

The use of experiential practices in science and engineering can be quite problematic when focusing on the foundational biology concept of evolution by natural selection. For such a concept, the use of realistic inquiry activities, especially with an engineering purpose, is difficult given the inability to actually breed organisms over generations in the high school classroom. Technological modeling tools would seem a natural choice to allow students to examine and refine their ideas about evolution through analyzing experimental data. However, prior attempts at this approach with high school students have had limited success, as we review below. Studies are needed to uncover scalable and effective modeling-based curricula with engineering themes in high school biology that focus on developing foundational concepts in an experiential manner. This study seeks to fill the gap in this research area by studying the efficacy of a unit focused on having students experience the foundational concepts of evolution by natural selection and population change in terms of effects on student content learning and attitudes.

Literature Review

Learning Challenges in Evolution

Most biologists would agree that evolution is the major foundational concept in biology across grade bands (Dobzhansky 2013; Klymkowsky 2010; Tansey et al. 2013). Evolution is key to understanding many other core concepts such as variation, genetics, and ecosystems (Tansey et al. 2013). Unfortunately, students have major difficulties learning this concept and harbor a number of alternative conceptions widely held across ethnic, cultural, and socioeconomic backgrounds (Bishop and Anderson 1990; Gregory 2009; Nehm and Schonfeld 2007). The most common alternative evolution-by-natural-selection conceptions students have at the undergraduate level are (1) inheritance of acquired traits, (2) loss of traits through disuse (i.e., when you no longer use a feature, it leaves the population), and (3) the origin of changes (e.g., that need for change directly causes speciation) (Nehm and Schonfeld 2007).

Other key concepts form the foundation for understanding evolution. For example, system thinking about populations, the ability to understand characteristics of and changes to groups of organisms, is necessary to fully comprehend the effect of evolutionary mechanisms such as natural selection. System thinking in population ecology involves several key concepts: populations have the ability to increase exponentially, population characteristics are described by frequency of occurrence in the population, and that population growth is limited by environmental factors such as predation and resource availability. If this systems thinking about populations is not understood well, it can lead to further problems during class instruction in evolution and population genetic concepts (Gregory 2009; Smith et al. 1995).

Student thinking about populations is often scattered with alternative conceptions. For example, many high school students think that an ecosystem’s resources are limitless (Brody and Koch 1990) and some undergraduate non-science, technology, engineering and mathematics (non-STEM) majors think that populations exist in either a constant state of growth or a constant state of decline (Munson 1994). Students also have a tendency not to consider that populations have special needs, and thus they consider that populations do not have any special effects on the environment and easily co-exist with other populations (Munson 1994). Griffiths and Grant (1985) discovered that students think two populations in a community would only affect each other if they shared a predator-prey relationship. This does not portend well for developing system thinking about populations that would support students’ emerging concepts of evolution.

Instructional Interventions in Evolution

Meta-analyses of inquiry instruction studies in K-12 science learning found a medium effect size for inquiry methods in comparison with traditional methods of instruction for science learning overall (Furtak et al. 2012). Thus, constructivist inquiry methods where students design experiments, analyze data, and generate explanations about science phenomena based on evidence should support student learning in the areas of population ecology and evolution via natural selection. These methods should allow students the ability to activate their fine-grained knowledge during the analysis of data so that these resources and their application in the specific contexts they are considering can be evaluated and refined to support their progress towards a more expert model of evolution or population ecology (Hammer et al. 2005; Maskiewicz and Lineback 2013; Smith et al. 1993).

However, these content areas are pragmatically difficult to implement as hands-on inquiry activities because the core phenomena happen over long periods of time, rendering experiments with actual organisms unfeasible. For natural selection topics, there have been attempts to at least use some active learning techniques such as discussion and cooperative problem solving, but these studies either lacked a comparison (Robbins and Roy 2007) or showed only modest gains compared to a comparison group (Nehm and Reilly 2007). There have been even fewer active learning studies that focused on population ecology. One such study that lacked a comparison and was focused on systems thinking about populations discovered that the use of hands-on units did not produce much shift in student thinking (Hogan 2000).

Engineering in Biology

While there is a call for the use of engineering practices within the context of K-12 classrooms (Quinn et al. 2012), there is a lack of studies researching the effectiveness of the engineering design process to teach core biology concepts in high school, especially evolution and population change topics. Engineering approaches in science class have produced student gains in physical science concept knowledge (Apedoe et al. 2008; Mehalik et al. 2008; Sahin 2010; Zeid et al. 2014), scientific reasoning skills (Silk et al. 2009), mathematical understanding (Hernandez et al. 2014; Schuchardt and Schunn 2016), understanding of cell processes (Ellefson et al. 2008), and engagement during science instruction (Doppelt et al. 2008). Thus, the use of an engineering approach could possibly further facilitate conceptual gains as well as provide further exposure to engineering practices.

Models and Modeling

One approach to including inquiry and engineering in teaching of evolution is the use of models. Model-based pedagogies use scientific models to make sense of science. These curricula can either present existing science models to students which are then used in problem solving (Passmore and Stewart 2002) or they can ask students to use data analysis to construct a scientific model to be predictive in problem solving situations like an engineering challenge (Schuchardt and Schunn 2016).

There are many ways to define models and modeling. The National Research Council (NRC) framework distinguishes between mental and conceptual models (Quinn et al. 2012). Mental models are implicit, and not visible to others except through expression in external representations such as conceptual models. Conceptual models, on the other hand, are the explicit representations that allow the science phenomena in question to be more understandable and predictable for students. These explicit representations include graphs, computer simulations, diagrams, analogies, mathematical equations, and physical models. The explicit representations can allow us to obtain insight into the character of a student’s mental model. The NRC framework articulates scientific modeling practice as the development, refinement, and use of scientific models (Quinn et al. 2012). Thus, modeling in science is often intertwined with the use of other science and engineering practices such as analyzing and interpreting data, using mathematical and computational thinking, designing solutions, constructing explanations, and supporting conclusions with evidence (NGSS Lead States 2013).

Modeling has been shown to be very powerful in producing gains in student problem solving ability in physics and genetics (Malone 2008), as well as conceptual gains in physics (Liang et al. 2012), chemistry (Dori and Kaberman 2012), and genetics (Schuchardt and Schunn 2016). The development of multiple model representations may be contributing to the success in student learning, since the use of multiple representations has been associated with an increase in the depth of students’ understanding (Tsui and Treagust 2003) and expertise in problem-solving (de Jong et al. 1998; Malone 2008; Tsui and Treagust 2007). The use of models for evolution and population ecology instruction have not yet yielded strong results, as detailed in the next two sections.

Conceptual Modeling of Existing Data About Natural Selection and Population Ecology

Several model-based biology efforts focused on conceptual model evaluation of existing data have been attempted at both secondary and college levels. Passmore and Stewart (2002) developed a 9-week-long high school model-based biology course in natural selection centered upon case studies called Modeling for Understanding in Science Education (MUSE). The course did not have the students’ actively producing biological models but instead had them evaluating alternative models of evolution such as Lamarckian and Darwinian. For example, students would be given data about a classic evolution situation such as similar coloration of monarch and viceroy butterflies and asked to use a natural selection model to determine why the coloration existed. The students were not participating in developing, refining, or modifying the models of natural selection they studied. While the authors briefly mentioned that students developed a richer understanding of natural selection, because the focus of the article was to present a curriculum, supporting empirical data was not provided and therefore the specific effects of model comparison on student learning of natural selection has not been clearly established.

At the college level, Dauer et al. (2013) studied a model-based biology class that focused on students developing models of genetics, evolution, and population growth while making connections between them. Students’ model correctness increased throughout the semester, and the gains of the lower tier of students were the greatest. However, this study did not have a comparison group and did not investigate students’ ability to apply the models to solve problems.

Modeling Simulation Tools for Natural Selection and Population Ecology

A number of researchers have turned to the use of simulations in order to facilitate the use of models and modeling in classrooms. Simulations give students the ability to construct a model of natural selection based on data, to test that model’s parameters, and to modify the model relative to predicted outcomes. The use of technology in the form of simulations such as STELLA (Korfiatis et al. 1999) and computer agent-based modeling environments such as NetLogo (Wilensky 1999) hold promise in assisting the learning of students in model-based courses. However, past results using such approaches have not had strong success, as summarized below.

One research team used a combination of MUSE and NetLogo to instruct one class of 8th graders during 7 weeks (Xiang and Passmore 2015). The 7-week time frame consisted of 3 weeks of learning to code in NetLogo in a computer class while using the MUSE curriculum in a science class, followed by 4 weeks of testing their model via computer simulation programming. The students’ task was to program a NetLogo simulation of adaptation to test the natural selection model developed in class. During the programming of their simulation, the students were impeded by their limited programming knowledge and at times were unable to develop a simulation to test their model parameters. Therefore, at the end of the study, some students still had fragmented or incomplete models of natural selection (Xiang 2011; Xiang and Passmore 2015).

Wilensky and Reisman (2006) completed an in-depth study of one secondary school student developing a predation model using NetLogo. Wilensky and Resiman determined that while the final model constructed predicted the observations in the lab setting, it was not consistent with what is found in nature, highlighting challenges of using such tools to help students develop accurate models of fundamental biological phenomena.

The Web-based Inquiry Science Environment (WISE) has a unit with an embedded simulation focused on fish populations and how they reacted to habitat changes. This unit was used in a 7th grade classroom to help students develop a model of natural selection. An evaluation study was conducted that focused mostly on how three teachers scaffolded the use of the WISE program in their classrooms over 7 days (Donnelly et al. 2016). The students were given a pre- and post-test that revealed a gain in conceptual understanding, which was largest for those students with lower prior knowledge. However, there was no comparison group so it is not possible to evaluate the effect of instruction involving the WISE simulation as opposed to the effect of simply receiving direct or inquiry-based instruction. The group discovered that the teacher who used more specific guidance targeted at helping students compare specific features of the data produced the greatest gains. This result highlights the need in modeling classrooms to have strong teacher support in order to assist student learning.

A simulation known as EvoBeaker Darwinian Snails Lab allows students to develop experiments to determine the effect on snail shell thickness over time (Meir et al. 2005). The effect of this simulation on alternative conceptions in natural selection in the context of multiple college classrooms was studied by Abraham et al. (2009). Their analyses of pre- to post-test changes demonstrated that while the students had a reduction in the use of common alternative conceptions, they largely could not explain the process of natural selection. Unfortunately, there was also no comparison group.

In one study, STELLA was used to teach population growth at the college level (Korfiatis et al. 1999). This simulation is a graphical-based simulation that allows users to modify parameters and see their effect on populations while testing different models. The research team determined that while the students using STELLA demonstrated a stronger understanding of the mathematical representation of population growth, their conceptual knowledge was weaker than the comparison group.

As a group, these studies have illustrated the importance for curriculum design of not having distracting students with learning to program, as well as the importance of teacher in guiding the student attention on interpreting data. In the description of the curriculum in the “Methods” section, we will show how we incorporated those lessons into the design of the Modeling Instruction in Biology with Excel (MBI-E) curriculum. In addition, there is the challenge of classroom time. One pragmatic constraint for using computer simulations is the time required to learn programming languages or complex modeling tools. The majority of schools in the USA are increasingly feeling the pressure to produce on high stakes testing which limits the time they have available for these two subjects. Finally, the use of the computer simulations discussed above would require the downloading of new programs on district computers that usually requires special permission, which can be a major barrier to teacher adoption (Iriti et al. 2016).

Modeling in Science vs. Modeling in Engineering

As reviewed above, modeling in science and modeling in engineering has the potential to support student learning in all sciences. The framework developed by the NGSS Lead States (2013) specifically highlights the scientific and engineering practices that should be routinely used by students in the classrooms in the USA. In addition, the NRC Framework (2012) specifies how engineering and science share numerous practices such as developing and using models, planning and carrying out investigations, and engaging in argumentation from evidence. Thus, science and engineering practices have many overlaps including that of the use of models. However, the use of models is for different primary purposes (NRC 2009). In engineering, the models are primarily used to help solve problems, while in science they are primarily used to answer questions (although engineers sometimes also use models for answering questions and scientists sometimes use models to solve problems). The use of engineering in the science classroom could be considered the ultimate deployment of a student’s newly developed scientific model (i.e., the ability to solve a problem).

Attitudes About Science

Engagement and interest in a subject have been shown to affect academic performance (Singh et al. 2002) as well as future activities and persistence (Ainley et al. 2002). Thus, it is important to develop engaging materials that both support student learning and positive attitudes that will shape future STEM career choices. In addition to interest and engagement, students’ beliefs (competency beliefs or self-efficacy) in their abilities to be able to perform successfully in an area of study have shown to be predictive of achievement (Schunk et al. 2008). While competency beliefs might also develop with inquiry-oriented tasks that better support ability development, there can be a difference between perceived abilities and actual abilities. Lawson et al. (2007) showed that college students’ scientific reasoning abilities were more predictive of their science success than competency beliefs, but competency beliefs can shape future STEM career choices. It is not known how the use of modeling tools will influence student interest and competency beliefs, although inquiry-oriented activities often have higher levels of engagement (Hernandez et al. 2014; Potvin and Hasni 2014; Zeid et al. 2014).

Research Questions

In conclusion, the majority of model-based or technological modeling studies in biology have not included comparison groups, they have taken large amounts of class time, the technology would need special training for teacher use, and little attention has been focused on the use of model representations. The majority of these curriculum attempts have focused on either evolution or population ecology. Evolution by natural selection relies on systems thinking about populations; thus, there is a large overlap in the areas of student difficulty in these two conceptual areas. Therefore, this study suggests that curriculum in one should include instruction in the other. The use of population ecology and systems thinking about populations to ground an instructional unit in evolution should lead to student learning in both areas. In addition, competency beliefs and interest in science have not been assessed in model-based curriculum reforms. Hence, this study attempts to fill a void by developing a modeling curriculum that is grounded in the use of easy-to-distribute Excel modeling tools, has an engineering design theme, and takes no more than 2 months to cover evolution, natural selection, and population ecology.

The goal of this design study was to (1) develop a modeling-based biology curriculum unit that incorporated simple but powerful Excel modeling tools at the center of the learning environment, and (2) examine the effects of the unit on student performance in terms of science motivation and understanding key concepts in population growth and natural selection. A quasi-experimental research study was conducted within demographically diverse classrooms, using pre-/post-content assessments and post-test motivational surveys. The following research questions guided the study regarding the effects of an Excel-based inquiry unit compared with more convention biology instruction:

  1. 1.

    Do students develop greater conceptual understanding of natural selection and population ecology?

  2. 2.

    Do students develop fewer alternative conceptions of natural selection and population ecology while utilizing a greater number of model representations?

  3. 3.

    Do students develop greater interest in and self-confidence in science?

Methods

Participants and Settings

In the USA, the largest groups of high school science learners are enrolled in introductory, non-honor biology courses. Therefore, teachers engaged in this level of biology education were recruited into the study. Implementation teachers were recruited through regional partners who provide professional development services to science teachers. Comparison teachers were recruited to conservatively match (i.e., be at the same or higher levels) implementation teacher characteristics (such as years teaching and prior education) and student characteristics (such as socioeconomic status, grade level, and high stakes test performance). Teachers were recruited from diverse school types (independent, religious, public) and areas (urban, suburban, rural) across two Midwestern states to insure generalizability across prior student preparation in science and mathematics as well as across varying prior exposure to ecological environments that could ground reasoning about population change and ecology.

The 11 teachers from ten high schools within the study taught over 400 students—255 students were in the implementation group and 169 students were in the comparison group. While the students were all enrolled in 1st-year regular-level biology classes, given some variation in where this class is placed in the high school curriculum, the grades ranged from 9th to 11th grades (with students therefore ranging from 14- to 17-year-olds). A detailed list of the matching characteristics as well as their significance can be found in Table 1. The schools were not significantly different on percentage of minority students and ACT 2013 science scores but were significantly different on percent of economically disadvantaged students, length of class periods, and years of experience. The significant differences favored the comparison students, as they were less economically disadvantaged and teachers with more years of experience. In other words, the comparison groups were purposely biased in a number of ways such that the traditionally taught students should have outperformed the MBI-E students simply based on demographics of students and teachers.

Table 1 Characteristics of comparison and implementing schools
Full size table

Teacher Training

Unit implementation by the MBI-E teachers was supported via teacher professional development workshops led by the research team. The comparison teachers were exposed to similar professional development workshops that focused on the effects of the materials rather than classroom implementation. The implementing teachers attended an average of 5.5 h of professional development workshops, while the comparison teachers attended an average of 4.1 h. The professional development for both conditions utilized model-based curricula as the instructional centerpiece, and included an introduction to the use of modeling and excel simulations for developing scientific models. The professional development for the MBI-E teachers also included an overview of the unit they would be implanting and an introduction to the teacher educative materials that went along with the student materials. Thus, both set of teachers had a similar exposure to ideas about models of natural selection and modeling in science, but only the MBI-E group was given the detailed materials to implement the MBI-E curriculum (Supplementary Appendix 1).

The educative teacher materials (Davis and Krajcik 2005) provided to implementing teachers included specific guidelines for use, introduction of the Excel modeling tools, instructional targets of specific activities, and possible student ideas raised by the activities. These educative teacher materials were based upon observing usage of the curriculum in pilot schools and were situated in a modeling inquiry-oriented context. The teacher educative materials also included instructions for implementing cooperative groups and leading productive whole-class discussions on these specific biology topics (see Supplementary Appendix 2).

The Curriculum Unit

An iterative development approach was employed to create the curriculum and assessments. The curriculum unit was designed to take place within the average amount of time that high school teachers reported spending on evolution and natural selection coverage in a prior survey, about 6 weeks. This unit covered not only evolution and natural selection but also population ecology topics within the time frame usually allotted to only evolution and natural section. The curriculum, both student and teacher educative materials, went through multiple rounds of revision and testing with many teachers and students. For example, completely different cover story (eradicating landmines in Laos using explosive-detecting fireflies) and approach to using Excel (for curve fitting) were initially developed and tested with a large group of teachers. Based on an evaluation of strengths and weaknesses of two full iterations of that unit, a new unit was developed, switching to a stinkbug/invasive species storyline and a transparent Excel simulation focused on model testing and representation matching. However, many representational tools successfully developed in the prior round were adapted to the new unit (e.g., initial by-hand simulations and graph drawing techniques and classroom discussion strategies). The revised unit was first piloted in two classrooms in the Midwest region of the USA. Based on classroom observations, teacher input, and post-test responses, refinements to this revised approach were made. The revised version of the stinkbug curriculum was then tested in the study reported in this paper.

The stinkbug unit is divided into two sections with different conceptual foci (population ecology and natural selection) that are situated within an overarching bioengineering design challenge (eradicating invasive stinkbugs using parasitoid wasps) that motivates learning by providing a real-world context. Figure 1 details the flow of the unit. The first section introduces population ecology concepts such as patterns in population data and predator/prey relationships culminating in the student production of a model of population growth grounded in systems thinking about populations. Observing lack of directed growth towards a desired outcome in the basic population growth model creates a desire among the students to determine how to selectively breed specific traits in a population for the design challenge using selective pressure. Using their knowledge of selective pressure then guides the students in developing the model of natural selection. Table 2 lists main target concepts. The curriculum was designed so that students would be positioned to activate their knowledge resources related to population ecology and natural selection, and make necessary refinements to the knowledge resources activated using Excel-based modeling simulations within the overarching context of a bio-engineering challenge. This design allows students to refine their initial ideas about these topics by focusing students’ attention to the need to rationalize their predictions based upon the data generated in their modeling simulations. Therefore, students build on their prior knowledge while building a more coherent and expert-like model of natural selection and population ecology (Hammer et al. 2005; Maskiewicz and Lineback 2013; Smith et al. 1993). Finally, after developing the two main models, students utilize refined understanding to develop a method to answer the bio-engineering design challenge. As can be seen in Fig. 1, at each point in the curriculum, the students are constantly referring back to the design challenge to motivate and conceptually connect the activities.

Fig. 1
figure 1

Flow of curriculum unit

Full size image
Table 2 Unit target concepts
Full size table

The unit is described in detail in the supplemental materials in order to illustrate the different ways the MBE computer simulations are used to support model development and revision. In addition, the detailed descriptions show how students could develop deepening levels of conceptual understanding through iteratively developing and refining their models of population growth and natural selection. Students develop models using the Excel modeling tools throughout the unit. For example, when developing the population growth model, students move to the use of the MBE computer simulation when they discover the difficulty of manipulating by hand paper simulations through multiple generations. The Excel modeling simulation allows them to have the ability to manipulate variables such as birth rate and death rate while quickly having the ability to look at both the predicted and observed growth rate graphs (i.e., model output) of the organism in question (in this case, wasps). Figure 2 is a screen shot of both the variable input page as well as the output graph page. As shown in the figure, students’ predicted output does not match the observed growth. Thus, students need to focus on the variables that they can manipulate, and reason through what they need to do in order to produce a narrowing in the differences between the prediction and the observed growth rates. In order to develop a robust model, students continually adjust the input variables until their observed and predicted graphs match while producing an explanation about why these adjustments are made.

Fig. 2
figure 2

Excel modeling tool sample—population growth. a Population growth input page. b Population growth output page

Full size image

Research Tools

Conceptual Assessment

The unit post-test consisted of 21 multiple-choice questions, and ten focused on natural selection (see example in Fig. 3), seven on population ecology (see example in Fig. 4), two on speciation, and two on mathematical randomness. It was designed to take no more than one class period in any of the schools (i.e., at most 40 min). The post-assessment was administered to all students after each class had covered both population ecology and evolution. The questions were designed to assess conceptual knowledge in these areas using multiple representations while making use of common alternative conceptions as response alternatives. Examples of the multiple representations are shown in Figs. 5 (pictorial), 6 (verbal), and 7 (graphical).

Fig. 3
figure 3

Natural selection example question using a pictorial representation

Full size image
Fig. 4
figure 4

Population growth example question using a verbal representation

Full size image
Fig. 5
figure 5

Population ecology example question using a graphical representation

Full size image
Fig. 6
figure 6

Mean percentage posttest scores (and SE bars) by instructional condition for total score and topic subscores

Full size image
Fig. 7
figure 7

a Average scores by instructional condition based on question representational focus for natural selection questions. b Average scores by instructional condition based on question representational focus for population ecology questions

Full size image

The natural selection test questions involved previously released Trends in International Mathematics and Science Study (TIMMS) and National Assessment of Educational Progress (NAEP) questions and items adapted from the Conceptual Inventory for Natural Selection (CINS, Anderson et al. 2002). CINS questions needed to be adapted because they had been designed for college-level students and its multi-paragraph reading level was deemed much too difficult for the participants in this study. Questions and alternatives address specific alternative conceptions such as the following: that new traits arise in or occur as a single event (Gregory 2009; Soderberg and Price 2003), that individuals who do not change will die (Gregory 2009), or that all members of the same species are nearly identical (Anderson et al. 2002; Soderberg and Price 2003).

Because there has not been a widely used population ecology assessment validated for research, the team adapted several questions from TIMMS, NAEP, and standard high school textbook questions as the basis for the construction of the population ecology assessment items. For example, questions and response alternatives address the idea students either believe that populations are in a constant state of growth, either negative or positive (Munson 1994), or that populations experience no change (Magntorn and Helldén 2007).

The entire assessment was given to pilot students as a post-test; based upon their answers, changes were made to question stems and answer alternatives. The internal consistency (KR-20) of the post-test’s multiple-choice questions was 0.80, sufficient for use in making comparisons between and within subject groups. For assessment purposes, the concept assessment was designed so that subscores could be produced for analysis. The two subscales of interest in this study are the Natural Selection subscore and the Population Ecology subscore.

Two equivalent conceptual assessment pre-tests were constructed by dividing the post-test in half due to the desire of the participating teachers not to use two full classrooms periods for testing. The pre-tests were balanced based on concept focus and difficulty level as determined by the pilot.

Attitude Assessment

Students were given a post-attitudinal survey to determine their level of general interest (labeled as fascination in the instrument) with science (13 items, e.g., “I wonder about how the world works,” with choices from every day to never; Cronbach alpha = 0.90), and their competency beliefs with tasks involving science content (ten items, e.g., If I did my own project in an after-school science club, it would be,” with choices ranging from excellent to poor; alpha = 0.89). The fascination survey determines a student’s interest in mastering scientific sills and concepts as well as their scientific curiosity and interest in science situations (Lin and Schunn 2016). The competency survey assesses a student’s beliefs about their ability to be successful in pursuing scientific endeavors (Vincent Ruz and Schunn 2017). The items on both surveys involved 4-point Likert scales. The highest level was coded as a 4 and the lowest level was coded as a 1. Each survey was given to a subsection of the class to be able to fit both the conceptual and attitude assessments into one class period.

Analysis

Standardized z-scores were determined for each conceptual pre-test as two separate pre-tests were produced for the study. Statistical significance was determined using ANOVA on post-test scores by condition. The conceptual assessment scores were analyzed for statistical significance in terms of the overall score as well as on the natural selection and population ecology subscores, using ANOVA. All assumptions (i.e., normality, homogeneity of variance, and independence of cases) were met. The post-attitudinal scores were analyzed for significance between the two groups using ANOVA on the subscores. Despite the difference in sample size for the implementing and comparison differences, ANOVA was chosen for the statistical analyses, because (1) the assumption of homogeneity of variance was met, and (2) ANOVA is only sensitive to unequal sample sizes when the homogeneity of variance assumption is violated (Harwell et al. 1992). Statistical significance between conditions was determined using t test for all post-test attitude surveys.

Results

Conceptual Assessment Pre-test

It was determined via ANOVA that the standardized z-scores at pre-test were not significantly different between conditions (F(1519) = 0.14, p = 0.71). Additional significance testing was completed to determine if the natural selection and population ecology subscores on the pre-test were equivalent. These additional tests showed that the implementing and comparison groups were also equivalent at pre-test on the natural selection (F(1474) = 1.24, p = 0.27) and population ecology subscores (F(1474) = 1.17, p = 0.28).

Conceptual Assessment Post-test

Quantitative Analysis

The post-test was not only analyzed in its totality but also subdivided in order to determine differences in understanding between the MBI-E implementing and comparison groups in the areas of population ecology and natural selection (see Table 3). An ANOVA showed significant differences between groups on all comparisons (p < 0.05). The mean scores and standard error bars can be seen in Fig. 6. Because within each of the two main focus areas each answer stem was directly connected to prior knowledge observed in literature or teacher observations, analysis of the answer selections would allow for determination of the types of knowledge resources still being used by each group of students. Table 4 shows the percentage of students in each condition who did not experience a conceptual change and still embraced the main alternative conceptions discussed in the literature review. The conceptual assessment post-test results demonstrate that the incorporation of Excel modeling tools and engineering practices in the biology classroom helped students demonstrate a greater conceptual understanding of both natural selection and population ecology topics.

Table 3 Post-assessment scores and p values by condition
Full size table
Table 4 Percentage of students holding alternative conceptions on post-test for comparison and implementing students
Full size table

Qualitative Analysis

In order to develop a more in-depth understanding of the conceptual differences between the two groups, an analysis was conducted of specific answer choices selected on the conceptual assessment. The assessment questions focused on verbal descriptions as well as pictorial and graphical representations of the concepts surrounding natural selection and population ecology. If one focuses only on the total use of the three representations highlighted, the MBI-E group consistently outscores the comparison group on all every representational usage. However, when considering average representational success of the two groups on questions focused on the two main subscores, differences start to emerge as seen in Fig. 7a (natural selection) and Fig. 7b (population ecology). Since there was a dramatic difference in representational success between these three topics, the 12 questions with differences in solution success (between 6 and 22%) were analyzed for common themes (see Table 5). The remainder of the post-test questions had similar performance between groups; no question showed an advantage in favor of the comparison group.

Table 5 Characteristics of items showing higher performance in the implementation students
Full size table

The themes that arose from the analysis varied by the concept being assessed. For natural selection, the questions most problematic for the comparison classes were ones grounded in graphical and pictorial representations. The comparison group scored 8 to 15% lower on all the natural selection problems focused on graphical representations, demonstrating a limited ability to symbolize natural selection using these representations. The comparison group’s individual question scores showed that while they performed better on some problems that were pictorial in nature, they still scored lower than the implementation group on 100% of these types of problems (i.e., 6, 6, and 16%, respectively). The test items that demonstrated differences in solution success of less than 6% were mostly dealing with verbal representations of natural selection. When asked to work in the verbal representational space, the comparison group only demonstrated a difference in performance on one question but that difference in score with the MBI-E group was 22%. This question dealt with acquired traits and Lamarckian evolution. Thus, the comparison group still had not fully confronted their prior conception that acquired traits could be passed down to descendants.

For population ecology questions, in contrast, the representations that seemed to be the most problematic for the comparison group were ones that were grounded in verbal representations. Only half of the population ecology assessments items grounded in graphical representations had a success rate in favor of the implementation group, with only one graphical question showing a group difference greater than 6%. This may seem a bit odd until one considers that population ecology in a typical classroom is heavily grounded in the use of graphical representations of population change such as J-curves and S-curves. The comparison groups’ difficulty with the verbal representations may have resulted from difficulty in mentally translating their learned graphical representations into verbal ones posed in the questions and vice versa. The comparison students also seemed to have difficulty verbally explaining population growth changes. For example, when asked what would happen to the population when confronted with limited or limitless food sources, the comparison group thought a population with limited food would continue to grow steadily but when focused with a limitless supply that the population would double then grow steadily. That is, the comparison group could accomplish graph matching but had difficulty explaining in depth why these graphical changes occurred.

While the comparison and implementing group could both graphically pick out which graphical representation was associated with the predator and prey (as shown in Fig. 5), the comparison group was not able to predict what might occur verbally in the predator prey situation shown in Fig. 4, scoring 10% lower than the implementing group. Thus, the implementing group was more easily able to move back and forth between graphical and verbal representations, as shown by their higher versatility in successfully maneuvering between multiple representations.

Attitudinal Changes

The post-fascination and competency belief attitudinal survey scores were analyzed between groups for statistical significance using t tests (see Table 3). The mean differences between the implementing and comparison condition showed that students in the implementing group were more fascinated by science after instruction then those in the comparison group (t(204) = 3.34, p = 0.001). Therefore, the MBI-E curriculum unit fosters an increase in interest in science, presumably related to either the use of engineering design problems with salient consequences or the increased exposure to conceptual complexity.

The students in the implementing condition demonstrated lower competency beliefs about science overall than students in the comparison group at the end of the unit (t(219) = 3.31, p = 0.001). Presumably, the greater rigor/challenge required by the MBI-E unit produced greater actual competencies in the students but its difficulty appeared to reduce perceived competencies. We take up possible causes and remediation strategies in the general discussion.

General Discussion

Students Develop Greater Conceptual Understanding of Natural Selection and Population Ecology

The Model-Based Inquiry unit using a modeling simulation grounded in Excel (MBI-E) supported significantly higher accuracy on a conceptual assessment for implementing students vs. matched-demographic, traditionally taught students. Overall, the students taught using scientific modeling simulations based in Excel scored one third of a standard deviation higher on the conceptual assessment than the students taught using more traditional methods. Further, when the conceptual assessment is subdivided into natural selection and population ecology subscores, the implementing students demonstrated a stronger grasp of both concepts taught. That is, even though the 6-week unit attempted to teach both population ecology and natural selection in an inquiry manner while also providing students the ability to explore a bioengineering project, both models showed large gains.

These findings demonstrate that the two conceptual areas of natural selection and population ecology, both difficult for students to master (Bishop and Anderson 1990; Gregory 2009; Nehm and Schonfeld 2007), can be effectively taught using scientific and engineering modeling strategies as recommended by the NRC Framework (Quinn et al. 2012). To our knowledge, this is the first study that demonstrates that scientific modeling in natural selection and population ecology can produce significant gains in conceptual understanding above and beyond that obtained through traditional instruction while taking only a moderate amount of time. In addition, these findings demonstrated that the use of simulations developed using a common spreadsheet tool easily available to teachers (i.e., Excel) is a viable solution to the obstacle to modeling presented by biological phenomena that occur over long periods of time. Thus, a computer simulation can enable biology students to engage in modeling practices in order to develop sophisticated scientific models such as that of natural selection.

Students Develop Fewer Alternative Conceptions Using MBI-E

When looking at the answer stems chosen for multiple questions on the conceptual assessment, it was clear that modeling instruction students harbored fewer alternative conceptions about natural selection and population ecology than the traditionally taught students. The main alternative conceptions still displayed by the traditionally taught students were the ideas that acquired traits (i.e., such as the accidental loss of an appendage) could be passed down to descendants.

In the case of population ecology, the main alternative conception dealt with what happened in environments with limited or unlimited food supplies. While the majority of the modeling instruction students seemed to clearly understand what would happen to population size in these situations, the comparison group floundered. The majority of the traditionally taught students could not describe what would happen to a population experiencing an environment with unlimited food supplies nor what it would look like to have limited food supplies.

It is highly possible that these results occurred because the comparison groups did not have the ability to activate and refine their knowledge resources, make predictions, and then compare these predictions against data. That is, the MBI-E group may have been led to activate their knowledge resources and then were pushed to make sense of the differences between predictions and the data obtained from the Excel modeling simulations. Thus, while the members of the MBI-E group were able to revise and refine their models, the comparison group members never realized that they might need to refine theirs (Nokes-Malach and Mestre 2013).

If one of the main expectations of a biology course is to allow students to revise their alternative conceptions in order to develop a clear understanding of the models in question, it seems clear that modeling-based curricula in biology can help teachers move students towards this goal (Schuchardt and Schunn 2016). In the area of physics, other studies have also shown that high school model-based and modeling curricula often lead to more expert-like knowledge on the part of students (Malone 2008). However, this is the first study we know of that shows that using modeling techniques via computer simulation tools easily obtained and manipulated by teachers such as Excel can lead to these changes in a relatively short period of time.

Students Are More Versatile with Different Model Representations When Taught Using MBI-E

The traditionally taught students demonstrated they were mostly successful when focusing on one representation per concept area. In fact, the representation that they were most expert at manipulating for each concept area differed. When working with natural selection, the traditionally taught students were more adept at working with verbal representations. However, while answering population ecology questions, the comparison groups were much closer to the MBI-E’s scores when the questions focused on graphical representations (refer to Fig. 7b). The traditionally taught students were not able to move easily from one representation to another within the same concept area. It may be that in typical classroom practice teachers only focus mostly on graphical representations when working with population ecology but rarely focus on verbal and pictorial representations. However, during natural selection the practice is reversed with students rarely moving out of the verbal representation space into graphical or pictorial representations. The current study leads to the hypothesis that the traditional practice of focusing on single representations during instruction hampers students’ abilities.

The use of modeling in the context of the MBI-E curriculum unit allowed students to routinely grapple with multiple representations in these two biological content areas. Indeed, the curriculum encourages teachers to allow students to develop multiple pictorial representations in order to more easily connect the ultimate graphical representation to what is actually happening in the environment. This study design not only allowed for the determination that biology students taking the MBI-E curricular unit were more facile moving between multiple representations (e.g., pictorial, verbal, and graphical) of the concepts; it also did so via the use of a specially designed multiple choice post assessment. Thus, this study is one of the first to compare the use of multi-representations between groups using an objective assessment.

The ability to fluidly move between multiple representations during problem solving is a hallmark of expert behavior in multiple fields (Harrison and Treagust 2000). Thus, this study shows that the MBI-E curriculum moved students from a novice stance to a more expert-like stance in the context of these two biological models. While this has been demonstrated in other less messy fields such as physics (Jackson et al. 2008; Malone 2008), this is one of the first studies in high school biology to do so using quasi-experimental methods.

Students Develop Greater Interest

The modeling-taught students were significantly more interested in science than the traditionally taught students, supporting the claim that the MBI-E curriculum fostered an increase in science engagement. Interest in science has been linked to not only student engagement (Dorph et al. 2016; Sha et al. 2016) but also student decisions to participate in both in-school and out-of-school science learning opportunities (Sha et al. 2015). Thus, affecting fascination in science in a positive way could lead to an increase in the number of students choosing to enter STEM degree programs.

Science educators have a goal to also instill in their students the belief that they not only are able to succeed in diverse science activities but also have the science skills needed to do so. This study revealed a small lowering of science competency beliefs in implementing students. This is troubling since student competency beliefs or self-efficacy towards science has been linked to effort and persistence in the learning environment (Sha et al. 2015).

The lower perceived self-competency may be caused by the rigor of the MBI-E curriculum which requires students to design experiments, support their arguments with evidence, defend their conclusions, and fix experiments that might go awry. This is most likely the first time that they have been asked to wrestle with science in this manner which can be intimating, especially when they realize that these abilities are much more difficult then they originally suspected. On the other hand, it is rewarding to note that the more rigorous curriculum produced greater fascination with science than the standard practice even if students felt their abilities were stretched. It is very possible that if MBI-E students were to experience more than one modeling based unit, their self-competency beliefs might rebound and surpass that of the comparison group. The cautionary tale here is that teachers need to make sure that they use modeling methods in their classroom regularly in order to produce the maximum benefits, especially attitudinally.

Limitations

One of the limitations of this study is that students in both conditions only took an attitudinal post-survey. Of course, without pre- and post-tests it is difficult to demonstrate for certain that a modeling-based curriculum such as the MBI-E does produce significant gain in science interest or competency beliefs as we do not know for certain that student interest and competency between conditions were not significantly different. Therefore, this study only implies that modeling-based instruction may affect these attitudinal areas.

Concluding Remarks and Implications

This study determined that teacher use of spreadsheets as modeling tools does significantly increase student knowledge as well as shift alternative conceptions towards scientific ones in terms of population ecology and natural selection. In addition, model-based instruction increases student fascination in science that can lead to persistence in later science classes.

The study did determine that student competency beliefs in model-based classrooms over the short term might show declines over that of traditional classes. This is most concerning in the classroom if teachers choose to only utilize model-based curricula for a unit or two. We suspect that if model-based curriculum materials are used for a longer time, then student competency beliefs would rise over the long term as they realize that they have the cognitive tools to develop evidence-based scientific models. However, this is an area that needs further research in order to explore these effects on student motivation.