One serious issue impeding the improvement of STEM education in US schools is a culture of high-stakes testing (Nichols & Berliner, 2007). Maltese and Hochbein (2012) noted that the decline of science instruction and curriculum narrowing beginning in primary schools have inhibited efforts for STEM improvement, as teachers have focused more on preparing students for standardized testing in other subjects. While recent STEM reform initiatives promote integrating the subjects, technology and engineering are often left out (Johnson, Peters-Burton, & Moore, 2016; Maltese and Hochbein, 2012; National Research Council [NRC], 2011). To confront this “one literacy at a time” stratagem, many educators advocate for curriculum integration (Barcelona, 2014; Beane, 1995; Honey, Pearson, & Schweingruber, 2014).

In order for an integrated curriculum to be successful, it must be designed coherently and not be fragmented by a series of incoherent topics (Beane, 1995; Frykholm & Glasson, 2005; Shepardson, Niyogi, Choi, & Charusombat, 2009). A scope and sequence, goals, activities, and assessments must contain crosscutting concepts, practices, and core discipline ideas that are both discipline-field specific and interdisciplinary (Drake & Burns, 2004; Furner & Kumar, 2007; Jacobs, 1989; Shepardson et al., 2009). Agriculture is a topic that has a clear link between STEM subjects. It can provide relevance, authenticity, familiarity, and context in terms of demonstrating how the resources and products people consume daily are designed and engineered and pass through scientific and mathematical processes with environmental impacts on our well-being (Vahoviak & Etling, 1994). That authentic context can serve as the catalyst to transfer prior learning to new topics in STEM subjects (Rivet & Krajcik, 2008). While familiarity does not equate to literacy, agriculture can link together knowledge, skills, and attitudes and beliefs (KSABs) found in agriculture to those in each STEM subject. Literacy efforts in STEM fields advocate for the development of informed decision-makers (knowledge) who are sensitive to their impacts on the natural world (attitudes/beliefs) and can participate in discourse relating to public policy that influences their health, the environment, and the future (skills). Similarly, both STEM and agricultural educators struggle with students’ misconceptions and motivations when integrating subjects (Chumbley, Haynes, & Stofer, 2015; Stofer & Newberry III, 2017), so purposeful integration may be the key to KSAB development and student interest across topics.

Creating authentic learning environments and integrated, standards-aligned curricula are important in promoting agricultural and STEM literacy because they can deliver unfamiliar content in a coherent way (NRC, 1988). The purpose of this study was to develop and test the efficacy of a coherent and authentic, standards-aligned, project-based curriculum incorporating KSABs in STEM subjects around the topic of agriculture. Dewey’s “learn by doing” philosophy guided the construction of the Agricultural Literacy through Innovative Technology (AgLIT) curriculum to support education reform initiatives’ hopes of producing knowledgeable, inquisitive, and conscientious life-long learners (NRC, 2012; Partnership for 21st Century Skills [P21], n.d.). This ten-day curriculum included pre- and posttest knowledge and attitudes instruments, eight performance tasks, and a field trip to the agricultural education center where students engaged in an augmented reality game on iPads. Here, we describe the curriculum, present knowledge and attitudes outcomes from the study, and provide suggestions for developing integrated agriculture and STEM curricula.

Theoretical Framework

According to Jacobs (1989), integrated curriculum connects “more than one discipline to examine a central theme, issue, problem, topic, or experience” (p. 8) to combat fragmented subjects, instruction that is isolated from application, and the irrelevance of school in the real world. Etim (2005) added that curriculum integration’s student-centered pedagogical view would also address students’ learning styles and developmental needs, and connect interests to real-life issues. Integrated curriculum can improve the skills necessary for students to be successful twenty-first century learners, such as collaboration, engagement, real-world knowledge transfer, and critical, holistic thinking about social problems (Kellough & Kellough, 1999; Partnership for 21st Century Skills, n.d.).

Using Agriculture to Integrate STEM

While the NRC (1988) suggested that all students in K-12 leave school with some degree of agricultural literacy, many curricula that offer agricultural content still focus primarily on plants or food and nutrition, often leaving students with a lack of understanding of the importance and breadth of the agricultural system and its myriad interconnections within society (American Farm Bureau Federation [AFBF], 2017b; Meischen & Trexler, 2003). Accordingly, Dewey (1905) noted, if educators “[r]elate the school to life, all studies are of necessity correlated” (p. 107), and agriculture is a topic relatable to both STEM and our lives.

While most Americans are disconnected from farm living, agriculture can enhance general education curricula, increase literacy in multiple topic areas, and serve as the keystone to STEM integration (Knobloch, Ball, & Allen, 2007). Even though roughly 2% of the US population works directly on farms, the agricultural industry employs roughly 15% of the population in processing, packaging, distribution, sales, and marketing (AFBF, 2017a; Dimitri, Effland, & Conklin, 2005; United States Department of Agriculture [USDA], 2016). According to Knobloch and Martin (2002), “the agricultural, food, fiber and natural resources system is the largest employer in America and plays a significant role in the nation’s economy...[and] provides a major prospective context for learning in today’s schools and classrooms” (p. 12). Agriculture can be an integral part of curriculum, providing authentic and relevant materials and contexts for STEM learning (NRC, 2009; Powell, Agnew, & Trexler, 2008).

Scholars have encouraged integrating curricula and incorporating agriculture into other fields to increase both agricultural and STEM literacy (Powell et al., 2008). For instance, current science textbooks and curriculum programs often describe the roles of plants as part of the food chain, components of the ecosystem, or whose parts and processes are worthy of description and analysis (Vallera & Bodzin, 2016). However, allowing students the opportunity to virtually design a garden plot, grow plants from seeds, experiment with water, soil, and sunlight conditions, and mathematically calculate their healthy, edible outcome may likely help them to connect the KSABs involved in growing food to related STEM fields. Agriculture can unify STEM concepts and encourage “citizens [to] make informed choices as voters to support or oppose public policies on agriculture-related issues, such as genetically-modified organisms in food production, food safety, and on food security, environmental quality, and land use” (Malecki, Israel, & Toro, 2004, p. 1) and draw attention to “pesticide usage, soil and water conservation, and other environmental concerns” (Law, 1990, p. 5).

Project-Based Learning Strategies

According to Thomas (2000), project-based learning (PBL) “is a model that organizes learning around...complex tasks, based on challenging questions or problems, that involve students in design, problem-solving, decision making, or investigative activities; give students the opportunity to work relatively autonomously over extended periods of time; and culminate in realistic products or presentations” (p. 1). With the focus on authenticity, critical thinking, innovation, and hands-on exploration of a central or driving question, students gain a deeper understanding of material and “connected networks of concepts” derived from using and representing “information in many different ways” (Marx, Blumenfeld, Krajcik, & Soloway, 1997 p. 342). Project-based approaches can serve as a means for gaining students’ attention, motivating them through real-world challenges, building their confidence by encouraging their success and scaffolding their prior knowledge, and providing them with a satisfying experience that is fun, interesting, and relevant to their lives (Hmelo-Silver, Golan Duncan, & Chinn, 2007). Students can explore and inquire about the natural world, which will lead them to ask better questions, analyze data more thoroughly, and solve authentic, real-world problems as lifelong learners and responsible consumers (Lee, 2004). PBL increases higher-order thinking skills, motivation, and engagement, and problem-solving skills by evoking students’ natural curiosity for exploration, goal directed behavior, and self-efficacy (Savery, 2006) and may create a more innovative and competitive workforce equipped with STEM skills.

Using Technology-Enhanced Learning as the Means of Agricultural Delivery

The “business as usual” approach of didactic instruction using adopted textbooks and materials to guide curriculum has several disadvantages compared to technology-enhanced environments and may not produce long-time learning (Healy, 2000). According to Petraglia (1998), educational technology has allowed educators to incorporate more authentic, constructivist thinking in the classroom by mirroring real-world activities. Lord and Orkwiszewski (2006) noticed “rapidly evolving technologies in education have created students who demand instantaneous feedback and involvement....[that] are no longer content to sit passively through a lecture or laboratory activity...[but] be engulfed in it” (p. 342; see also Barab & Luehmann, 2003; Tapscott, 2009). Clark (2001) also argued that it is not just technology, but the related technology-integrated curriculum changes that increased learning. Technology may provide students with virtual fieldtrips and video-conferences with experts, as well as activities, projects, and problems (Wiggins & McTighe, 2005) that provide questions and challenges for students to investigate using authentic data. The integration of PBL strategies with appropriate technologies can also provide instruction that appeals to different learners using novel stimuli to gain attention while enhancing interest and motivation (Gagné, 1985).

Research Purpose and Questions

The goals of this study were to (1) add to the existing knowledge about the nature of agricultural and STEM literacy of upper-primary students, (2) create a fully STEM-integrated agricultural literacy curriculum to increase literacy in agriculture and STEM fields, and (3) test the efficacy of that curriculum. Fully STEM-integrated lessons, project-based activities, and formative and summative assessments surrounding the topic of agriculture were created and built in an iBook application accessible with an iPad. The curriculum was designed by an instructional designer and was validated by three STEM curriculum experts, while addressing the following research questions:

  1. 1.

    Knowledge Research:

    1. a.

      How do students receiving an agricultural literacy curriculum score on a knowledge assessment of science, technology/engineering, and agricultural content compared to a control group between pretest and posttest?

    2. b.

      How do students receiving an agricultural literacy curriculum differ on individual subscale measures of science, technology/engineering, and agricultural content compared to those in the control group after controlling for pretest subscale scores?

  2. 2.

    Attitudes/Beliefs Research:

    1. a.

      How do students receiving an agricultural literacy curriculum alter their individual attitudes/beliefs about science, technology/engineering, and agriculture compared to a control group following the implementation of the curriculum?

    2. b.

      How do students receiving an agricultural literacy curriculum alter their individual attitudes/beliefs about science, technology/engineering, and agricultural content compared to those in the control group after controlling for pretest subscale scores?

Methods

Participants

The study took place in a large urban school district in the northeastern U.S. using the technology-enhanced, standards-aligned, and project-based AgLIT curriculum in fourth-grade classrooms. In order to gain a representative sample, all fourth-grade teachers from one of the district’s primary schools were invited to participate in the study. Ninety-five students and four fourth-grade teachers from the district consented to participate in the study. The school enrolled 626 students, 89.1% of which qualified for free or reduced lunches. The students identified as Hispanic (72.0%), black (11.0%), white (9.7%), multiracial (6.2%), and Asian (0.8%). Four teachers and their classes were randomly selected into either the treatment or control group. Forty-seven students in two classes were assigned to the treatment group and 48 students in two classes were assigned to the control group. The treatment teachers received professional development and support materials for the AgLIT curriculum prior to implementation.

Research Design

The study employed a pretest/posttest quasi-experimental comparable control group design (Campbell & Stanley, 1963) to analyze data from the KnowASTE assessment and the ThinkASTE measure (discussed below). The treatment group received the curriculum over ten-consecutive days, which will be described in detail later, and the control group received regular classroom instruction from the district-adopted Full Option Science System (FOSS) curriculum that included no additional agricultural content following the pretest administration. All students completed the pretest and posttest KnowASTE and ThinkASTE measures.

Design-Based Research Methods

The AgLIT curriculum was developed through several iterations utilizing design-based research (DBR) methods. DBR methods encourage collaboration between researchers and teachers throughout the design, analysis, and implementation stages of project development to “produce meaningful change in contexts of practice” (Design-Based Research Collective [DBRC], 2003, p. 6). Utilizing DBR allowed for a better understanding of which contexts best supported the inclusion of agricultural literacy, and the “[r]eliability of findings and measures can be promoted through triangulation from multiple data sources, repetition of analyses across cycles of enactment, and use (or creation) of standardized measures or instruments” (DBRC, 2003, p. 7). Well-designed materials that include avenues for failure, feedback, iterations, and that reduce cognitive load on students can increase instructional efficiency and sustain student learning. Teachers that participated in this study were involved in the review and revisions of the materials, provided iterative feedback and modifications to the tasks, and offered suggestions prior to, during, and following the curriculum’s development and implementation. Collaboration between the researchers and teachers was vital to the fidelity of implementation during the study.

Curriculum Design and Integration.

Agriculture is a complicated and often misunderstood subject (Leising, Igo, Heald, Hubert, & Yamamoto, 1998). Children and adults alike generally do not know where their foods and fibers originate, the steps taken to deliver finished products to retailers, or the importance agriculture plays in society as well as the rest of the world. In order to adequately convey these messages, lessons and activities were designed as authentically and engagingly as possible to give students realistic and relevant learning experiences that make large-scale global connections between agriculture and people and encourage students to learn on their own, from others, and through inquiry. Important design principles for implementing interdisciplinary curriculum, incorporating PBL strategies, and engaging students with technology were utilized.

Design principles geared toward implementing interdisciplinary curriculum included (1) making integration explicit, (2) attending to students’ disciplinary knowledge, (3) attending to the social aspects of learning, and (4) supporting the development of interest and identity when integrating curriculum (Honey et al., 2014). Agriculture and STEM subjects were integrated after crosscutting KSABs in each subject were identified, essential questions were developed, and PBL strategies were used to construct authentic and contextual driving questions, lessons, and activities students might encounter in similar real-world situations. The AgLIT curriculum was designed to encourage the following: (1) asking questions and defining problems; (2) developing and using models; (3) planning and carrying out investigations; (4) analyzing and interpreting data; (5) using math and computational thinking, (6) constructing explanations and designing solutions; (7) engaging in arguments from evidence; (8) obtaining, evaluating, and communicating information, and others.

AgLIT is aligned to A Framework for K-12 Science Education (henceforth, Framework; NRC, 2012), Common Core Standards (NGACBP, 2010), National Agricultural Literacy Outcomes (henceforth, NALO; Spielmaker, 2013), Food and Fiber System Literacy Framework (henceforth, FFSL; Leising et al., 1998), and Next Generation Science Standards (henceforth, NGSS; NGSS Lead States, 2013). It was designed to encourage both STEM and agricultural literacy learning goals and KSABs. Using the Understanding by Design framework (Wiggins & McTighe, 2005), the curriculum engaged students by offering integrated content and performance tasks guided by the PBL driving question: How will you help Farmer Kathy prepare her products for sale at the farmers’ market? Farmer Kathy is the farmer students met through videotaped lessons opening the curriculum and during a fieldtrip to the agricultural education center following the unit.

The curriculum incorporated eight performance tasks designed to help students prepare for the farmers’ market. Lessons and activities were designed as authentically as possible to give students realistic and relevant learning experiences that make large-scale connections between agriculture and STEM and encourage students to learn on their own, from others, and through inquiry. It was important that design principles geared toward thoughtfully incorporating PBL and technology were utilized in the construction of all materials. Those principles involved situating learning in realistic and relevant environments, making thinking visible to students, cooperation and collaboration, authenticity, and hands-on designing. PBL strategies and innovative technologies allowed for the development of content, tasks, and assessments to include those principles and engage students outside of the “business as usual” approach of didactic instruction. Background content knowledge was provided through an interactive iBook equipped with text, videos, simulations, and formative questions. Aside from having content presented through the use of PBL and innovative technology in the interactive iBook, the AgLIT curriculum included additional technologies and design challenges to engage students involving ArcGIS, Google Earth, augmented reality, engineering and presentation design, and math and scientific data analysis. Engaging students in authentic tasks using technologies experts in the field might use can make learning more meaningful. Technology was used as a tool to disseminate content and tasks and was studied as a subject in integrated STEM.

All content and assessments were reviewed by a panel of three science and environmental educators, the participating agricultural educational center, and a large animal veterinarian to ensure content accuracy, alignment with the targeted content understandings, and construct validity. A primary teacher reviewed the full curriculum to ensure acceptable readability for the targeted students. Participating teachers reviewed all materials and assessments as well to provide feedback through design-based research methods.

Knowledge, Skills, and Attitudes/Beliefs Measures.

The curriculum included a pre- and posttest knowledge assessment that measured science, technology/engineering, and agricultural literacy (henceforth, KnowASTE) and a pre- and posttest Likert-scaled attitudes/beliefs measure surrounding the same three constructs (henceforth, ThinkASTE). Mathematics was not included as a subscale content measure on the knowledge or attitudes instruments because math anxiety, if present, often affects test scores in timed situations (Ashcraft & Moore, 2009), which may influence students’ overall results. However, mathematics was the focus of two performance tasks and was included in many other tasks as well (explained further in the next section). Each of the performance tasks was assessed with a valid and reliable rubric; however, the final project-based task, which was a culmination of the prior days’ activities, was used in the final analysis of the pilot-test with results presented elsewhere. The curriculum was implemented during ten-consecutive days of in-class instruction with two separate days for pre- and posttests (see Table 1 for the scope and sequence).

Table 1 Curriculum scope and sequence
Full size table

The pretest/posttest KnowASTE assessment included 27 multiple-choice questions (worth 1 point each) created by the author or adapted from multiple sources, where select questions were modified to align to this curriculum’s lessons and activities. The adapted items came from the Engineering is Elementary curriculum (Museum of Science, 2014), the grades 4–5 Food and Fiber Systems Literacy Test (Igo, Leising, Frick, Hubert, & Malcolm, 1999), and Utah’s Agriculture in the Classroom (http://www.agclassroom.org/teacher/agknow.htm) assessment geared toward students in grades 3–5. The KnowASTE instrument was aligned to NALO (Spielmaker, 2013), FFSL (Leising et al., 1998), Common Core (NGACBP, 2010), and NGSS (NGSS Lead States, 2013) standards with three subscale content measures—science, technology/engineering, and agriculture. Technology and engineering were combined into one construct, as many of their applications in agriculture overlap. The KnowASTE assessment allowed for total scores that ranged from 0 to 27. Each of the three subscale content measures contained 9 items that allowed for scores that ranged from 0 to 9. Prior to the study, the KnowASTE was tested with students of varying ages and a split-halves reliability test was run to determine the instrument’s internal consistency reliability after respondents’ answers were coded as either correct or incorrect. This test yielded a Spearman-Brown’s reliability coefficient of .808 (N = 76) for all items. Cronbach’s alpha measures for each predetermined construct were gathered to determine internal consistency reliability and support construct validity. The science subscale yielded a Cronbach’s alpha score of .623 (N = 76), the technology/engineering subscale score was .633 (N = 76), and the agricultural subscale had a Cronbach’s alpha of .706 (N = 76). These coefficients were acceptable for internal consistency reliability (George & Mallery, 2001).

Similar to the knowledge assessment, the attitudes ThinkASTE measure included 27 Likert-style statements adapted from myriad sources that encompassed the three subscale content measures—science, technology/engineering, and agriculture. The adapted items came from Engineering and Science Attitudes Assessment (Museum of Science, 2010), the University of Montana’s Geotechnology in the Classroom Project (Crews, 2008), and the 1986 nationwide Farming in American Life questionnaire (Jordan & Tweeten, 1987). Again, technology and engineering were combined. Students’ total possible scores ranged from 27 to 81, since the Likert measures included numeric values of 1 (“I don’t agree”), 2 (“not sure”), and 3 (“I agree”). Each subscale content measure again contained 9 items that allowed for scores ranging from 9 to 27. Higher scores on the ThinkASTE and its subscale constructs indicated more positive attitudes/beliefs. A split-halves reliability test was run on this instrument prior to this study to determine the internal consistency reliability producing a Spearman-Brown’s reliability coefficient of .839 (N = 61) for all items. Cronbach’s alpha measures were gathered to determine internal consistency reliability and support construct validity for each of the subscales. The science subscale yielded a Cronbach’s alpha score of .802 (N = 61), the technology/engineering subscale was .663 (N = 61), and the agricultural content subscale had a Cronbach’s alpha of .721 (N = 61). These coefficients were considered acceptable measures of internal consistency reliability (George & Mallery, 2001). Both the KnowASTE and ThinkASTE measures are included in the online supplemental materials.

The Performance Tasks.

Throughout the curriculum, students participated in eight performance tasks; they received task sheets and written instructions surrounding a guiding question and authentic scenario a farmer may encounter. Each activity focused primarily on an aspect of STEM surrounding a realistic agricultural project, although other STEM skills were needed for each activity’s completion. For instance, in a science task, students learned about the importance of protecting natural resources, such as soil and water, when they investigated the types and properties of soil needed for growing indigenous plants employing scientific investigation. Additionally, students timed and measured soil permeability when water was added using math skills. In a mathematics activity, students calculated measurements if they were to double a cheese-making recipe, practiced fractions when identifying different ways to make the same measurement when lacking appropriate tools, and practiced multiplication and division when adding rennet tablets to their recipes. In an engineering task, students were challenged to design a profitable garden and appropriately place it virtually (using Google Earth) on the farm that could produce native fruits and/or vegetables for the farmers’ market—something the farmer must consider every season when rotating and planning crops. In the culminating project (Sustainable Marketing Plan, Parts 1 & 2), students had to mathematically analyze market trends and calculate sale prices for appropriately grown products at the farmers’ market, create a virtual brochure of their goods for sale, design a scaled model of their future farm stand, and present their marketing plans to a real farmer. Students began by interpreting data from charts and graphs embedded in a website discussing farmers’ market trends. They then looked a current market prices for items they were looking to sell at their farm stand and created sales prices that allowed them to generate revenue but remain competitive. Students then developed their plan and designed their brochures and models.

The PBL curriculum was designed to encourage the development of agricultural and STEM literacy through the connected content, tasks, and assessments. The learning tasks allowed students to gain skills and connect newfound knowledge to real-world understandings of where their foods and fibers originate (NRC, 2012) and the processes they go through before reaching the consumer. These tasks were assessed with rubrics and are discussed elsewhere.

Procedures

The study was approved by the researchers’ institutional review board and the researchers were trained to ethically work with human subjects. Informed consent was gathered in writing prior to the dissemination of the curriculum, which included both parental consent and student assent forms. All parents and students asked agreed to participate in the study. KnowASTE and ThinkASTE pretests were disseminated 2 weeks before the first day of classroom instruction following receipt of the forms.

The 48 students in the control group received regular classroom instruction from the district approved curriculum that included no additional agricultural-specific content following the pretest administration. Their science administration was unaltered and the two teachers continued with their “business as usual” instruction. The district employed the Full Option Science System (FOSS) curriculum. FOSS is a research-based program geared toward promoting active learning and engaging students in authentic inquiry-based scientific investigation (What is FOSS?, n.d.). While treatment group students received the AgLIT curriculum, control group students investigated environmental factors in the Environment module, which consisted of exploration into how organisms’ structures allow them to survive in their environments (Environment Overview, 2019). A teacher-made unit test was administered at the end of their instruction.

The 47 students in the treatment group then received the ten-day curriculum module. The researcher and doctoral student were in the classroom to assist the teachers with project administration of the hands-on activities and serve as technology support throughout the module. The curriculum was implemented in the students’ 50-min science class periods and activities were completed during 50-min math periods. During each day of implementation, students began by reading background content related to the featured agricultural topic in the iBook, viewed embedded videos or simulations, and answered formative assessment questions. Students were then split into groups and were given an authentic scenario relating to the day’s topic that drove their investigation through the STEM-based performance task. Students worked together to solve the challenges but completed their own sheets. The activity completed each day prepared students for the culminating, fully STEM-integrated, project-based performance task that students presented to a farmer on the final day of the unit. Both posttests were given following the final day of classroom instruction.

Prior to data analysis, scores from the instruments were calculated. Statistical analyses were run to test the research questions to determine group differences following the curriculum’s implementation. Two-way mixed analyses of variance (ANOVAs) and multivariate analyses of covariance (MANCOVAs) were used to analyze data from the KnowASTE assessment, as well as the ThinkASTE instrument. All statistical tests were conducted using Statistical Package for the Social Sciences (SPSS 21.0). KnowASTE assessment scores were calculated by adding all of the points earned from correctly answering questions, generating total scores that ranged from 0 to 27. Scores from each of the three constructs (science, technology/engineering, and agriculture) were tallied to determine where any changes occurred in the individual constructs. The same method was used when analyzing scores from the ThinkASTE measure and addressing the secondary research questions. The total ThinkASTE measure scores were calculated by adding the scores from each of the 27 Likert-scale items to generate total possible scores ranging from 27 to 81. Tallies were generated to identify the two groups’ subscale content measure construct scores.

Results

Of the 95 students that consented and participated, 80 completed both the KnowASTE and ThinkASTE instruments and were included in the final data analysis. This section highlights the results from each research question. Overall differences in pretest and posttest scores on the KnowASTE and ThinkASTE instruments between the treatment and control groups are discussed, as well as specifically where those changes occurred (science, technology/engineering, and agriculture constructs). Descriptive statistics from the instruments are listed in Table 2.

Table 2 Evaluation score means, standard deviations, and percent changes by group and class
Full size table

It is important to note that treatment group students scored higher on the initial pretests for knowledge (M = 11.40, SD = 3.84) and attitudes (M = 64.90, SD = 4.41) than did the control group students for knowledge and attitudes respectively (M = 9.74, SD = 2.85; M = 61.03, SD = 5.64), and class B in the treatment group scored higher than all other classes in the study. While it is advantageous to use mixed ANOVA to study the groups’ changes over time (Stevens, 2009), the impact of the initial pretest differences may skew the model’s outcomes and an additional test was added to take those differences into consideration. To control for this initial difference in mean scores on the two pretest instruments, analyses of covariance (ANCOVAs, where pretest scores adjust the means of the posttest scores) were run to supplement the mixed ANOVAs. Since intact classroom groups that differed on their pretest scores were included in this study, using covariates (the pretest scores) can reduce some of the initial systematic bias and provide for a “fairer comparison” between the two groups (Stevens, 2009, p. 288). It can also be assumed that a linear relationship exists between the pretest and posttest scores on each of the instruments, so utilizing ANCOVAs is a wise choice to determine more equitable outcomes.

Overall Knowledge Acquisition

For the treatment group, the effect size (Cohen’s d) for the KnowASTE measure was .816, a large effect size. To test whether the AgLIT curriculum increased students’ knowledge of agriculture, science, and technology/engineering as a whole, a two-way mixed ANOVA was conducted to investigate whether the treatment and control groups’ scores differed between the pre- and posttest KnowASTE and precisely how those groups differed. Prior to running the mixed ANOVA, assumptions of normality, homogeneity of variance, and equal variance and covariance matrices were tested and met. The results of the ANOVA showed a significant group difference, F(1, 78) = 23.47, p < .001, partial η2 = .231, indicating that treatment group’s scores were statistically significantly different than the control group’s scores, and there was a significant difference between the two tests, F(1, 78) = 18.57, p < .001, partial η2 = .192. There was also a significant interaction between the tests and the groups, F(1, 78) = 17.91, p < .001, partial η2 = .187. This effect demonstrated that pre- and posttest scores differed between the two groups (see Table 3). Based on the means and the interaction plot, the treatment group scored higher on both the pretest and the posttest than the control group.

Table 3 Two-way mixed analysis of variance for KnowASTE pre- and posttests by group
Full size table

A one-way analysis of covariance (ANCOVA) was also run to determine the difference between the treatment and control groups’ KnowASTE posttest scores while controlling for the groups’ initial pretest score differences. Preliminary analyses evaluating the assumptions of normality, independence, and homogeneity of variance, based on Levene’s test of equality of error variances (p = .349), were conducted and met. Linearity and homogeneity of regression slopes indicated that the relationship between the pretest covariate and the posttest did not differ significantly as a function of the groups, F(1, 76) = .001, p = .981. The pretest covariate had a significant effect on posttest scores, F(1, 77) = 35.27, p < .001, partial η2 = .314, and the results from the ANCOVA indicated that there was a significant group difference in posttest scores between the two groups after controlling for pretest scores, F(1, 77) = 38.91, p < .001, partial η2 = .336. See Table 4 for the results from the KnowASTE ANCOVA. After the curriculum implementation, the treatment group’s adjusted posttest mean scores (M = 13.93, s.e. = .41) were statistically significantly higher (p < .001) than the adjusted mean scores for the control group (M = 10.21, s.e. = .43) indicating significant knowledge gains as a result of the implementation.

Table 4 Analysis of covariance for KnowASTE posttest scores by group
Full size table

Acquisition of Knowledge in Science, Technology/Engineering, and Agriculture

It was important to find out which specific content knowledge constructs—science, technology/engineering, and agriculture—changed following the curriculum implementation. A one-way multivariate analysis of covariance (MANCOVA) was conducted to examine the differences between the treatment and control groups on a set of three dependent variables that measure science, technology/engineering, and agriculture. KnowASTE pretest scores were included as a covariate in the model to reduce the amount of systematic bias that could arise from the differences among the two groups’ initial pretest scores. The data were evaluated with regard to meeting the statistical assumptions of multivariate normality (establishing univariate normality and bivariate normality), equality of covariance matrices of the dependent measures, homogeneity of regression, and significance of regression prior to conducting the MANCOVA.

The results from the MANCOVA determined that when controlling for KnowASTE pretest scores, there was a statistically significant multivariate difference between the treatment and control groups on the three subscales of science, technology/engineering, and agriculture (Wilks’ λ = .644, F(3, 75) = 13.834, p < .001). Univariate ANOVAs were conducted to analyze the groups’ differences in each of the three subscales separately. Univariate ANOVA results found significant group differences on all three measures, science (p = .001), technology/engineering (p < .001), and agriculture (p < .001). Bonferroni post hoc comparisons of the group means were run to determine the two groups’ means on the three outcome measures. When controlling for pretest scores, the adjusted posttest science mean scores for the treatment group (M = 4.45, s.e. = .22) were significantly higher (p = .001) than the control group’s (M = 3.32, s.e. = .23). Adjusted mean scores from the technology/engineering construct were also higher (p < .001) for the treatment group (M = 4.62, s.e. = .20) than for the control group (M = 3.53, s.e. = .21). Similarly, adjusted mean scores for the agriculture construct were also significantly higher (p < .001) for the treatment group (M = 4.86, s.e. = .20) than for the control (M = 3.36, s.e. = .21). Students in the treatment group increased their knowledge of science, technology/engineering, and agriculture following implementation.

Overall Changes in Attitudes and Beliefs

Similar analyses were conducted to determine whether overall attitudes and beliefs about science, technology/engineering, and agriculture were altered following the implementation of the AgLIT curriculum. For the treatment group, the effect size (Cohen’s d) for the ThinkASTE measure was 1.042, a large effect size. A two-way mixed ANOVA was conducted to investigate whether the treatment and control groups’ scores differed between the pre- and posttest ThinkASTE and precisely how those groups differed. Prior to running the mixed ANOVA, the assumptions of normality, homogeneity of variance, and equal variance and covariance matrices were tested and met. ANOVA results showed a significant group difference, F(1, 78) = 26.41, p < .001, partial η2 = .253, indicating that treatment group’s scores were statistically significantly higher than the control group’s scores, and there was a significant difference between the two tests, F(1, 78) = 26.17, p < .001, partial η2 = .251. There was also a significant interaction between the tests and the groups, F(1, 78) = 6.96, p = .010, partial η2 = .082 indicating that pre- and posttest scores differed between the groups (see Table 5). The results, means, and interaction plot indicate treatment students’ attitude improvements as a result of the implementation.

Table 5 Two-way mixed analysis of variance for ThinkASTE pre- and posttests by group
Full size table

Another one-way ANCOVA was run to determine the difference between the treatment and control groups’ ThinkASTE posttest scores while controlling for the groups’ initial pretest score differences. Preliminary analyses evaluating the assumptions of normality, independence, and homogeneity of variance, based on Levene’s test of equality of error variances (p = .521), were conducted and met. Linearity and homogeneity of regression slopes indicated that the relationship between the pretest covariate and the posttest did not differ significantly as a function of the groups, F(1, 76) = .398, p = .530.

The ThinkASTE pretest covariate had a significant effect on posttest scores, F(1, 77) = 42.90, p < .001, partial η2 = .358, and the results from the ANCOVA indicated that there was still a significant group difference in posttest scores between the two groups after controlling for pretest scores, F(1, 77) = 17.88, p < .001, partial η2 = .188. See Table 6 for the results from the ThinkASTE ANCOVA. Following the implementation of the curriculum, the treatment group’s adjusted posttest mean scores (M = 67.73, s.e. = .66) were significantly higher (p < .001) than the adjusted mean scores for the control group (M = 63.53, s.e. = .70). Treatment students’ attitudes/beliefs about science, technology/engineering, and agriculture were altered positively as a result of the curriculum’s implementation.

Table 6 Analysis of covariance for ThinkASTE posttest scores by group
Full size table

Changes in Attitudes and Beliefs in Science, Technology/Engineering, and Agriculture

A one-way MANCOVA was also conducted to determine the differences between the treatment and control groups on a set of three dependent variables that measured science, technology/engineering, and agriculture on the ThinkASTE posttest. ThinkASTE pretest scores were included in the model as a covariate to address any issues that could be caused by initial pretest mean differences between the two groups. Again, the data were evaluated using a MANCOVA analysis after the statistical assumptions of multivariate normality (establishing univariate normality and bivariate normality), equality of covariance matrices of the dependent measures, homogeneity of regression, and significance of regression were tested and met.

When controlling for ThinkASTE pretest scores, the results from the MANCOVA determined that there was a statistically significant multivariate difference between the treatment and control groups on the three constructs of science, technology/engineering, agriculture (Wilks’ λ = .781, F(3, 75) = 6.991, p < .001). Follow-up univariate ANOVAs were conducted to analyze the groups’ differences in each of the three constructs separately. Univariate ANOVA results found significant group differences on science (p = .006) and agriculture (p = .002), but not for technology/engineering (p = .065). Bonferroni post hoc comparisons of the group means were run to determine the two groups’ means on the three outcome measures. When controlling for pretest scores, the adjusted posttest science mean scores for the treatment group (M = 22.53, s.e. = .39) were significantly higher (p = .006) than the control group’s (M = 20.89, s.e. = .41). Adjusted mean scores from the agriculture construct on the posttest were also higher (p = .002) for the treatment group (M = 21.94, s.e. = .29) than for the control group (M = 20.52, s.e. = .31); however, there were no significant differences (p = .065) between the treatment group’s adjusted posttest mean scores (M = 23.26, s.e. = .40) and the control’s (M = 22.13, s.e. = .43) on technology/engineering. Treatment students had more positive attitudes/beliefs about science and agriculture following the AgLIT curriculum’s implementation, but showed little change in their attitudes/beliefs regarding technology/engineering.

Discussion

Recent curriculum reform initiatives have indicated the need to increase STEM literacy of students in the U.S.; however, several issues have stood in the way of such initiatives’ successes. Scholars suggest that STEM literate students are better equipped for 21st century jobs, make wiser decisions as consumers, and are able to participate in important discussions about their health, the environment, and technological advancements that may affect their futures (NGSS Lead States, 2013; NRC, 2011). Current curriculum reform initiatives encourage the integration of STEM subjects through crosscutting concepts, practices, and core discipline ideas to increase STEM literacy and provide students with more motivating learning experiences (Barcelona, 2014; Furner & Kumar, 2007; Honey et al., 2014; NRC, 2012). Creating authentic experiences surrounding agriculture can increase interest, engagement, achievement, and persistence, while decreasing the disconnection between the subjects (Furner & Kumar, 2007; Honey et al., 2014; NRC, 2012). Meaningfully integrated STEM curriculum surrounding agriculture can offer remarkable learning experiences if it is designed coherently, connected to pre-existing knowledge, presented authentically, and transferable to new situations far better than isolated facts presented incoherently (Honey et al., 2014).

This study tested the efficacy of a fully STEM-integrated, project-based curriculum designed to measure both agricultural and STEM literacy in hopes of developing knowledgeable and active lifelong learners able to participate in their futures as conscientious consumers. The AgLIT curriculum was evaluated as a complete product designed to (1) integrate STEM subjects around agriculture, (2) incorporate project-based learning strategies to increase KSABs, motivation, and engagement, (3) include educational technology to meet the needs of digital learners, and (4) utilize design-based research methods to involve teachers’ experiences and fulfill professional development requirements. Overall knowledge gains occurred in the treatment group on the KnowASTE assessment, as well as in the individual constructs—science, technology/engineering, and agriculture. Treatment students’ overall attitudes improved on the ThinkASTE and were found to be more positive toward science and agriculture specifically. Based on these findings, we offer the following recommendations about integrated design.

First, as mentioned, the push for integrated curricula is not new; however, it is often hard to accomplish. Utilizing an integrated curriculum can connect fragmented subjects through the application of real-world, authentic practice (Etim, 2005; Frykholm & Glasson, 2005). A coherent scope and sequence, thoughtful goals and objectives, and rigorous content, activities, lessons, and assessments must be constructed and implemented to meaningfully integrate curriculum. A unifying “keystone,” or topic, that can unite these things must be identified. The AgLIT curriculum included important crosscutting concepts, scientific practices, core discipline ideas, and aligned KSABs from STEM and agriculture. The unifying topic of agriculture engaged students to discover relationships between the subjects and transfer relevant KSABs.

Second, project-based learning strategies can encourage student engagement, motivation, and keep them invested in their learning. Goals must be identified upfront with outcomes that are satisfying enough for students to continue learning past the completion of a task. The driving question must be interesting and relevant and provide connections to real-world study through authentic challenges that scaffold students’ prior knowledge. The project-based learning strategies provided in the AgLIT curriculum tasked students with solving problems, creating products from raw materials (such as cheese, felt, and yarn), and exploring myriad solutions to different challenges each day. The challenge level of the tasks was appropriate to the students’ levels of ability but encouraged them to think more creatively and critically about information they may not have considered important in the past (i.e., agriculture’s importance in their lives).

Third, innovative educational technology, such as augmented reality, GIS, presentation slide decks, and mobile devices, can offer authentic, real-world examples and can illustrate complex, abstract concepts to can increase learner understandings and success with today’s digital learners (Barab & Luehmann, 2003; Tapscott, 2009). Technology must serve as a tool for integrating content learning and not as the main focus of the activity. In this study, the technology enhanced students’ experiences with STEM and agriculture with its innovation, interactivity, technological complexity, and applicability to their digitally-driven lives. Watching videos, answering questions, and reading about agriculture and farming in their iBooks kept students interested in the unit. The technology included novel and surprising events when students zoomed into the agricultural education center using Google Earth, studied the changes in agriculture over time using Web GIS, and saw augmented reality videos appear over pizza ingredients. Treatment students regularly requested the use of the iPads and expressed their excitement to “try new things” with the technology.

Fourth, providing teachers with professional development and opportunities to participate in design-based research strategies can improve the design of integrated curricula. Providing background content knowledge, materials with embedded scaffolding, and support materials about learner misconceptions can increase teacher confidence and lessen the time needed to administer project-based curriculum. Thus, instruction will not leave students unguided. Professional development and materials were provided in AgLIT supported and motivated teachers to implement an interdisciplinary, project-based learning curriculum with fidelity.

Limitations of the Study

Although there were many triumphs throughout the curriculum’s implementation, there were also several limitations. While the technology and iPad-enhanced nature of the curriculum was appreciated, its novelty was at times a bit distracting. Students were so excited to have iPads in the classroom the first 2 days that their enthusiasm took away from instruction and project time. Fortunately, design-based research strategies allowed for teacher consultation and modifications that removed the distracting features of the iPad use, such as providing a one-computer model for reading the background content and providing one or two iPads per group for the projects that required the technology with specific sharing and usage procedures.

There were other limitations of this study as well. First, the sample of students was gathered from one urban primary school, which will not allow for the generalization of findings to all upper-primary students. Students’ abilities and knowledge levels may not be representative of other demographically similar students. Secondly, extraneous variables related to teachers’ experience, pedagogical content knowledge, and classroom effectiveness might have influenced the findings in this study. Regardless of professional development opportunities offered prior to implementation of the curriculum, teacher differences may still have impacted the results. Thirdly, the implementation of the curriculum in a ten-consecutive day sequence might not ultimately impact STEM or agricultural literacy. Integrating the project-based curriculum throughout the entire adopted and implemented school curricula where they can enhance understandings of STEM and agricultural KSABs might be a more effective way of implementing the curriculum’s learning activities.

Conclusion

In a time when STEM integration and education reform initiatives are pressing, it is important that curriculum developers integrate subject areas coherently and consider designing programs that incorporate agriculture as the “keystone” for integration. Several studies have found that while teachers view agriculture favorably and recognize that it can be integrated into any subject (Knobloch & Martin, 2002), many still are not including it in their curriculum and classrooms (Bellah & Dyer, 2009). Numerous agricultural literacy materials are available for teachers’ use; however, teachers’ lack of interest or knowledge, their misconceptions or stereotypes, or lack of time due to curriculum time constraints and high-stakes testing may keep such learning materials out of the classroom (McReynolds, 1985).

With integration into STEM subjects, agriculture can augment classroom learning experiences by providing students with authentic, real-world examples that encourage deep thinking and knowledge acquisition (Balschweid & Thompson, 2000; Bellah & Dyer, 2009), while using innovative technology to motivate students with authenticity to benefit their learning. Regardless of Americans’ disconnect from farm life, the agricultural system affects every aspect of our lives and should be a strong focus of study because of that. Introducing agriculture in primary school is a good place to integrate it into US STEM curricula.

Utilizing agriculture as a vehicle for integration can increase students’ interest, participation, and knowledge in all of the fields simultaneously. The connections between the subjects are unmistakable and students’ achievements were identified by their scores on the knowledge assessment and attitudes measure. However, what was most important was finding that students not only learned about and had more positive attitudes toward STEM and agriculture, but that they enjoyed what they were doing and were motivated to each day of the unit. Hopefully, these students can now participate in conversations regarding where their foods and fibers originate and how the production processes relate to their lives.