Introduction

Climate change science topics are included prominently as disciplinary core ideas in US science education curriculum frameworks (American Association for the Advancement of Science 2007; National Research Council 2012). They are also viewed globally as being significant for both citizens and policy makers to understand in order to support decisions to mitigate anthropogenic effects of human-induced climate change (Intergovernmental Panel on Climate Change [IPCC] 2007). However, climate change science is a challenging topic for student learning. Research conducted during the past two decades has revealed that secondary students show knowledge deficits about specific climate change concepts and also have misunderstandings that are fundamental to understanding the science underlying climate change that pertains to the atmosphere, greenhouse effect, and the climate system (Bodzin et al., in review; Shepardson et al. 2012). Secondary students do not have a basic understanding about the volumetric composition of atmospheric gases (Bodzin et al., in review), greenhouse gas sources and their composition in the atmosphere (Boyes et al. 1993; Boyes and Stanisstreet 1993; Koulaidis and Christidou 1999; Leiserowitz et al. 2011; Punter et al. 2011), and the role of water vapor as a key greenhouse gas (Fisher 1998; Schreiner et al. 2005). Studies have also found that students have incomplete understandings of how the greenhouse effect works (Andersson and Wallin 2000; Koulaidis and Christidou 1999; Niebert and Gropengiesser 2013; Rye et al. 1997; Varma and Linn 2012) and have erroneous understandings about the cause–effect relationship between the greenhouse effect, global warming, and ozone layer depletion (Boyes et al. 1993; Gowda et al. 1997; Kilinç et al. 2008; Koulaidis and Christidou 1999; Liarakou et al. 2011). In addition, both students and adults often confuse the timescale between weather and climate (Read et al. 1994; Shepardson et al. 2009). This confusion may reflect misunderstandings about geologic time, an important component of paleocliamatology (Lombardi and Sinatra 2012).

Secondary students also have misunderstandings about anthropogenic sources that contribute to human-induced climate change. Studies have found that some students assume that all forms of pollution contribute to the greenhouse effect (Pruneau et al. 2001) and some students generalize that all air pollutants are greenhouse gases (Punter et al. 2011). Recent research has shown that secondary students have incorrect beliefs that acts of environmental stewardship that include cleaning up litter at the beach, protecting endangered species, reducing insecticides, reducing the use of Freon, and reducing nuclear energy use would help reduce global warming (Boyes et al. 2008, 2009; Daniel et al. 2004; Kılınç et al. 2011).

The design and enactment of the school curriculum may be an important factor for this lack of deep understanding of climate change science. While many of the studies noted above have focused on secondary students’ climate change conceptions, most lacked descriptions of the implemented school curriculum. In the USA, the commercially published science curriculum has been criticized for not covering topics sufficiently in-depth (Kesidou and Roseman 2002) and not helping students develop deep understanding of important ideas in science (Schwartz et al. 2008). There have only been a few published studies that have paid attention to the design of the curriculum to promote learner understanding of climate change concepts. Klosterman and Sadler (2010) reported that the implementation of a three-week unit on global warming targeting high school socioscientific issues resulted on average in more accurate understandings of global warming, the greenhouse effect, and the controversy and challenges associated with those issues. A seven-day global warming middle school curriculum that featured interactive Netlogo visualizations using a “knowledge integration framework” helped students to develop coherent climate change ideas of energy transfer and transformation concepts (Svihla and Linn 2011) and of the greenhouse effect (Varma and Linn 2012).

To address the issues with climate change understandings among middle-level learners, we developed a coherent 20-day climate change science curriculum for urban eighth-grade students using a geospatial curriculum approach. The curriculum integrates the geospatial technology Google Earth and other Web-based learning activities to support students’ understandings of climate change science. Teaching with geospatial technologies focuses learners on spatial analysis of patterns, relationships, and trends in geospatial data and phenomena (Baker et al. 2012). Many climate change effects on our planet have a geospatial component and can be visualized and understood through spatial analyses with geospatial technologies. Previous studies have found that integrating geospatial technologies within a coherent curriculum has been effective for enhancing middle-level students’ understandings of energy resources (Bodzin et al. 2013; Kulo and Bodzin 2013) and land-use change (Bodzin 2011; Bodzin and Cirruci 2009).

The goal of this study was twofold: (1) to examine the efficacy of a technology-integrated curriculum that used geospatial technologies to promote urban middle-level students’ understandings of important climate change science concepts; (2) to investigate factors related to both students and teachers that may account for the variance in students’ climate change knowledge achievement. This study took place in the context of a larger science education and urban school reform initiative to enhance the teaching and learning of environmental science topics in the school curriculum.

Geospatial Curriculum Design Approach

The climate change curriculum (henceforth Climate Change) used a geospatial curriculum design approach to learning that builds on the work of other successful technology-integrated curriculum projects (Bodzin 2011; Edelson 2001; Krajcik et al. 2008; Linn et al. 2004). This geospatial curriculum design approach supported teacher enactment by intentionally incorporating a curriculum framework and targeted design principles. An instructional model was used to guide the development of all learning activities, and materials for teachers were developed to assist with curriculum implementation in the classroom.

The curriculum framework included

  1. 1.

    Align materials and assessments with learning goals.

  2. 2.

    Contextualize the learning of key ideas in real-world problems.

  3. 3.

    Engage students in scientific practices that foster the use of key ideas.

  4. 4.

    Use geospatial technology as a tool for learners to explore and investigate problems.

  5. 5.

    Support teachers in adopting and implementing geospatial learning technologies and inquiry-based activities.

A series of design principles were used to promote geospatial thinking skills with the curriculum materials. The design principles included

  1. 1.

    Design curriculum materials to align with the demand of classroom contexts.

  2. 2.

    Design activities to apply to diverse contexts.

  3. 3.

    Use motivating entry points to engage learners.

  4. 4.

    Provide personally relevant and meaningful examples.

  5. 5.

    Promote spatial thinking skills with easy to use geospatial learning technologies.

  6. 6.

    Design image representations that illustrate visual aspects of scientific knowledge.

  7. 7.

    Develop curriculum materials to better accommodate the learning needs of diverse students.

  8. 8.

    Scaffold students to explain their ideas.

  9. 9.

    Use icons that portray the real-world concept.

See Bodzin et al. (in press) for a more detailed explanation of each design principle.

The geospatial investigations in Climate Change were developed with the virtual globe application Google Earth since it employs an interface that may be modified in ways to enhance initial data visualization displays for learners. The initial geospatial data visualizations for each activity were designed in such a way that they are quick and intuitive for both students and teachers to use, thus decreasing interface issues that have been previously reported by classroom use of other geospatial technologies such as GIS desktop platforms (Baker and Bednarz 2003; Bednarz 2004).

The instructional model included eight key elements to guide the development of each geospatial learning activity in the curriculum for promoting geospatial learning and reasoning skills. Each geospatial learning activity incorporates a sequence of instructional events that were based on current learning theories that are applied to the design task of promoting teaching and learning of science with geospatial learning technologies. The model included the following key elements:

  1. 1.

    Elicit prior understandings of lesson concepts.

  2. 2.

    Present authentic learning task.

  3. 3.

    Model learning task.

  4. 4.

    Provide worked example.

  5. 5.

    Perform learning task.

  6. 6.

    Scaffold learning task.

  7. 7.

    Elaborate task with additional questions.

  8. 8.

    Review activity concepts.

See Kulo and Bodzin (2013) for a more detailed explanation of each key element in the instructional model.

The curriculum included educative materials (Davis and Krajcik 2005) that were designed to promote and support teachers’ learning of important climate change science subject matter, geospatial pedagogical content knowledge, and geospatial thinking skills. It has been argued that teachers need to have a sound understanding of climate change concepts to effectively teach about climate change in the classroom (Lambert et al. 2012). However, research on preservice and practicing teachers’ climate change concepts and misconceptions have shown that teachers do not demonstrate adequate understandings of the climate change concepts and have the same misunderstandings reported in the studies of secondary learners (see, e.g., Arslan et al. 2012; Dove 1996; Groves and Pugh 1999; Khalid 2003; Lambert et al. 2012; Michail et al. 2006; Papadimitriou 2004). The educative materials were designed to call attention to important climate changes concepts so that teachers will be less likely to reinforce any students’ erroneous concepts or misunderstandings during curriculum implementation.

The curriculum also included baseline instructional guidance for teachers and provides implementation and adaptation guidance for teaching a variety of learners including reluctant readers, English language learners, and students with disabilities. The educative materials also provided teachers with rationales for instructional decisions. Teachers are known to draw on their own resources and capacities to read, make meaning, evaluate, and adapt curriculum materials to the needs of their students (Remillard 2005). If teachers understand the rationale behind a particular instructional recommendation, they may be more likely to enact the curriculum in keeping with the developers’ intent (Davis and Varma 2008).

The Climate Change Curriculum

The Climate Change curriculum included a coherent sequence of topics and learning activities designed to promote learner understandings about the atmosphere, Earth system energy balance, weather and climate, greenhouse gases, paleoclimatology, and environmental impacts of human-induced climate change. Table 1 outlines the 20-day instructional sequence of the curriculum. Five lessons during the 20-day curriculum used the geospatial curriculum approach with Google Earth and are described in more detail below. The curriculum also included lessons with a Web-based interactive carbon calculator and geologic time line, inquiry-based laboratories, demonstrations, readings, and modeling activities aligned to the curriculum learning goals.

Table 1 Instructional sequence of daily lesson activities
Full size table

In the first geospatial lesson (Day 2), students used Google Earth to explore global temperature changes during a recent 50-to 58-year period. Students explored, analyzed, and interpreted climate patterns of 13 different cities and analyze differences between weather and climate patterns. At the completion of this geospatial investigation, students understand that changes in climate are variable based on location and that the magnitude of observable climate change temperature is quite small. In the next lesson (Day 3), students used a Web site to learn about the four main Earth spheres and the composition of the atmosphere with an emphasis on greenhouse gases and the role that ozone plays in the troposphere and stratosphere. During the next series of lessons (Days 4–8), students explored and investigated concepts pertaining to Earth system energy balance including albedo, surface and atmospheric absorption, and reflection. In the next geospatial investigation (Day 9), students used Google Earth to investigate the climate patterns and geographic locations of seven cities in the USA to determine how latitude, elevation, proximity to bodies of water, and mountain ranges affect a location’s climate.

Student lessons (Days 10–11) then focused on the carbon cycle and the importance of greenhouse gases in our atmosphere. Students also learned about proxies and completed a paleoclimate reconstruction laboratory (Days 12–14). Students then used a Web-based geologic time line to examine temperature, CO2 concentration, and ice cover data to investigate how climate has changed during the last 715 million years (Day 15). At the completion of these activities, they understand that long-term climate patterns provide evidence for fluctuating CO2 concentrations.

In the next lesson (Day 16), students used a Web-based carbon calculator to determine their carbon footprint. Students examined their personal and household habits and choices in relation to their carbon footprint, identified which personal activities and household choices produce the most CO2 emissions, and then compared their carbon footprint to US and global emission averages. In this activity, students identified lifestyle changes they can make to reduce their carbon footprint.

The next three lesson included geospatial investigations that focused on using student analysis of evidence of climate change effects on our planet. Students investigated different geographic areas and populations affected by changes in climate patterns using map-embedded geospatial data using Google Earth (Day 17). Spatial data analysis included examining changes over time in the habitat ranges of pikas and polar bears, and examining time-sequenced photography of glaciated areas provides evidence of diminished snow cover over time.

In the next two-day lesson (Days 18–19), students used Google Earth to investigate the evidence of recent climate change. First, students navigated to the Caribbean Sea to view changes in coral reef distribution between 2001 and 2010. Some background text along with a coral reef image was provided to introduce students to the idea that increasing ocean temperatures associated with climate change is the cause of coral bleaching. Students then used embedded data layers in Google Earth to analyze and compare coral reef distributions in 2001 and 2010. They observed the areas of dramatic change to the southwest of Jamaica, where an observed coral reef community is no longer present in the 2010 spatial data overlay (see Fig. 1). Students then navigated to the Arctic to view changes in sea ice extent from 1979 to 2010. Students were presented with some background information about climate models and learned that modeling data from scientists show that increased ice sheet melt will lead to increasing sea-level rise. Students were then instructed to display and measure the continuous Arctic Ocean ice sheet during three time periods—1979, 1990, and 2010—with the Google Earth measurement tool. After students completed their ice sheet measurements, they calculated the area of the ice sheet for each year. The students’ area calculations showed that the sea ice extent in the Arctic has been decreasing since 1979.

Fig. 1
figure 1

Location of coral reefs near the island of Jamaica in 2001 (purple) and 2010 (yellow). The red box indicates a coral reef population that was present in 2001 but not present in 2010 (Color figure online)

Full size image

During the second day of the investigation, students were presented with the IPCC’s worst-case scenario of a 2-meter rise in sea level. They then used Google Earth to explore “future world scenarios” by examining the effects of a 2-meter rise in sea level on the existing landscape using the Google Earth elevation profile tool at six low-lying locations that could be affected by sea-level rise. Figure 2 displays the Wildwood, NJ, location that shows that much of the submerged land with a one-meter rise in sea level would be residential and marsh areas. Next, students observed 1-meter and 2-meter sea-level rise IPCC scenarios in the Chesapeake Bay watershed with embedded GIS data in Google Earth. The example of Assateague Island with a one-meter rise in sea level is shown in Fig. 3.

Fig. 2
figure 2

An example of the elevation profile tool displayed for Wildwood, NJ. The beginning and end of the Google Earth “path” are labeled. An arrow marks the location on the map and its corresponding location on the displayed elevation profile

Full size image
Fig. 3
figure 3

The example of Assateague Island shown with a 1-meter sea-level rise GIS data layer displayed. The eastern portion of the island, as indicated by the blue color, would submerge with a 1-meter rise in sea level (Color figure online)

Full size image

In the final lesson (Day 20), students explored strategies at personal and societal levels to help reduce atmospheric carbon emissions levels. They also explored how sustainable energy resources and carbon capture and storage methods can help reduce carbon emissions levels.

Research Questions

As noted earlier, we lack specific knowledge about the kinds of science curriculum experiences that may lead to student improvement in developing climate change science understandings with urban middle-level students. The primary aim of this study was to examine the effectiveness of a curriculum using a geospatial curriculum approach to promote climate change science understandings in an urban school district with eighth-grade students. This curriculum implementation study was guided by the following research questions:

  1. 1.

    Whether and to what extent can a geospatial curriculum approach promote climate change understandings with students in urban middle-level education?

  2. 2.

    What factors related to both students and teachers may account for students’ climate change knowledge achievement?

Methods

Participants

Twelve eighth-grade Earth and space science teachers, including one preservice teacher, implemented the Climate Change curriculum with their students during the 2011–2012 academic school year. The participants taught 1,060 students (ages 13–15) at all four middle schools in the same urban school district in the northeast region of the United States. The school district is located in a medium-sized city with a population of about 100,000 residents. Seven teachers were male and five were female. The teachers had a wide range of teaching experiences from a preservice science teacher to a teacher with 38 years of teaching experience. Content area certifications were quite varied and included general K-8 certifications, middle school science certifications, and secondary-level science content area certifications. One teacher taught science to two classes composed only of English language learners. The student population in the district’s middle schools was 49.1 % Caucasian, 34.7 % Hispanic, 12.1 % African American, 3.2 % Asian, and 0.9 % multiracial. Data attrition resulted from students who were not in school due to suspensions and truancy or did not return signed consent forms.

One teacher who was a member of the curriculum development team had pilot-tested the prototype version of Climate Change with her students during the previous school year. During October and November 2011, all teachers attended 3 days of professional development to become acquainted with the Climate Change learning activities.

Climate Change Assessment Measure

The climate change assessment measure included 28 multiple-choice items and 3 open-ended response items that aligned to current climate change knowledge goals stated in the recent US reform documents Climate Literacy: The Essential Principles of Climate Sciences (US Global Change Research Program 2009), Communicating and Learning about Global Climate Change (AAAS 2007), and the Framework for K-12 Science Education (NRC 2012). The assessment items were designed to measure climate change learning goals expected to be achieved by students by the completion of eighth grade. The items were reviewed by a panel of four earth and environmental scientists and science educators to ensure content accuracy, alignment with the targeted content understandings, and construct validity. A lead middle-level classroom science teacher in the school district reviewed the items to ensure acceptable readability for the targeted students. The entire assessment is included in the online supplemental materials, and selected items are included in “Appendix 1.”

The items were grouped into three subscales corresponding to three main climate change science topic areas:

  1. 1.

    Atmosphere, greenhouse effect, and climate system (AGC) [17 items]

  2. 2.

    Human-induced climate change (HCC) [7 items]

  3. 3.

    Paleoclimatology (PC) [4 items]

The multiple-choice items include distractors that address misunderstandings and knowledge deficits about climate change from the existing literature. The AGC items were designed to address students’ understandings about the definition and source contribution to the greenhouse effect, the role of water vapor as a key greenhouse gas, atmospheric composition, weather and climate, and essential features of the climate system. The HCC items addressed understandings about anthropogenic sources that contribute to climate change, environmental impacts, and solutions to climate change at a personal and societal level. The PC items focused on students’ understandings of paleoclimatology. Each multiple-choice item is assigned one point for a correct answer and zero points for an incorrect answer or blank response, yielding possible total scores ranging from 0 to 28. A criterion-based rubric was developed to score three open-ended questions using a 0- to 4-point scale (see “Appendix 2”). Thus, the possible scores on the open-ended items ranged from 0 to 12. See Bodzin et al. (in review) for a more detailed discussion about the climate change assessment measure. The reliability (Cronbach’s alpha) for all 28 multiple-choice items was 0.860, and the reliability for the three subscale topic areas was 0.785 for AGC, 0.613 for HCC, and 0.647 for PC.

Teacher Measures

The teachers completed two measures at the end of their third day of the Climate Change professional development sessions. These included (1) the measure for the 28-item multiple-choice items from the climate change assessment described above (possible total scores ranging from 0 to 28) and (2) the measure from Geospatial Science-Technological Pedagogical Content Knowledge [GS-TPACK] instrument (Bodzin et al. 2012). The GS-TPACK instrument was designed to measure teachers’ perceived knowledge of how geospatial technology interacts with their pedagogical content knowledge in ways that produce effective science teaching and student learning opportunities. The instrument includes 23 Likert-type items that are scored with a six-point scale of 1 (strongly disagree) to 6 (strongly agree), with the possible total scores ranging from 23 to 138. The evidence of validity was established by having the items reviewed by a panel of three researchers with expertise in geospatial learning in education. The reliability (Cronbach’s alpha) of the GS-TPACK instrument is 0.961.

Data Analysis

To address the first research question in a straightforward fashion, paired-sample t tests were conducted in IBM SPSS 21 to examine whether the mean scores of the climate change assessment items were significantly different between the pretest and the posttest. Since 47 students did not complete the pretest and 57 other students did not complete the posttest, 956 students were included in the analysis.

All open-ended response items were coded by one researcher, and 25 % of the items were coded by a second researcher independently. They were found to be in agreement 93 % of the time. Any discrepant cell placements on the rubric were resolved via discussions between the coders.

The second research question investigates whether the student- and teacher-related factors may account for the variance in students’ climate change posttest scores. The outcome variable was the students’ climate change posttest scores from the 28 multiple-choice items. The student predictor variables included gender (female = 1 and male = 0) and their climate change pretest scores on the same 28 multiple-choice items. The teacher-related factors included (1) teacher climate change content knowledge, (2) GS-TPACK, and (3) years of teaching experience.

Two approaches in total were tried to address the second research question. We first tried the ordinary least squares (OLS) multiple regression analysis using the blockwise method in IBM/SPSS 21 to investigate how well the teacher factors accounted for the variance in students’ climate change posttest scores, after controlling for students’ gender and pretest scores. The two student variables were entered in Block 1 and the three teacher variables were in Block 2.

Secondly, for the purpose of incorporating the data structure that students were nested within teachers, we conducted a series of two-level hierarchical linear models (HLM) (Raudenbush and Bryk 2002; Singer 1998) for the second research question. We did not specify the school level as Level 3 because we only had 4 non-randomly selected schools. The HLM were run using SAS PROC MIXED with the full maximum likelihood method.

We started from the unconditional means model (i.e., HLM without any Level-1 or Level-2 predictors), also referred to as one-way ANOVA with random effects. This simple model allows us to calculate the intraclass correlation coefficient (ICC) to determine how much variance in student climate change posttest scores is attributable to inter-teacher variation. The two-level regression equations consist of (1) Level 1 (student level; within-teacher): y ij = β 0j + r ij, expressing the posttest score for a typical student i of teacher j (y ij) by the sum of an intercept for the students within teacher j (β 0j) and a random error (r ij ); and (2) Level 2 (between-teacher): β 0j = γ 00 + υ 0j, expressing the intercept as the sum of a grand mean (γ 00) and random deviations from the mean due to teacher variation (υ 0j). Combined, the two-level HLM is given by y ij = γ 00 + υ 0j + r ij, where υ 0j ~ N(0, τ 00) and r ij ~ N(0, σ 2). We computed the ICC using the output for the variance among students within teachers (σ 2 = 22.81) and variance between teachers (τ 00 = 20.33). The resultant ICC 20.33/(20.33 + 22.81) = 0.47 indicates that 47 % of the variance in student climate change posttest scores is due to teacher heterogeneity. This substantial proportion of variance would be inappropriately ignored in OLS regression. Therefore, we further expanded the unconditional HLM by iteratively adding student- and teacher-related variables. After comparing the nested models based on -2 log-likelihood difference given their degree of freedom, we decided on the following random intercept and slope model, including (1) student gender and pretest scores at Level 1, and (2) at Level 2, the three teacher variables for the intercept and random effect terms for the intercept and the pretest slope:

$$ {\text{Level }}1:y_{\text{ij}} = \beta_{{ 0 {\text{j}}}} + \beta_{{ 1 {\text{j}}}} {\text{Pretest}}_{\text{ij}} + \beta_{{ 2 {\text{j}}}} {\text{Female}}_{\text{ij}} + r_{\text{ij}} , $$
$$ \begin{aligned} {\text{Level 2}}:\beta_{{ 0 {\text{j}}}} & = \gamma_{ 0 0} + \gamma_{01} {\text{Knowledge}}_{\text{j}} + \gamma_{02} {\text{GSTPACK}}_{\text{j}} + \gamma_{03} {\text{Years}}_{\text{j}} + \upsilon_{{0{\text{j}}}} , \\ \beta_{{ 1 {\text{j}}}} & = \gamma_{10} + \upsilon_{{ 1 {\text{j}}}} ,\,{\text{and}}\,\beta_{{ 2 {\text{j}}}} = \gamma_{20} . \\ \end{aligned} $$

Combined, the two-level HLM is given by

$$ \begin{aligned} y_{\text{ij}} = & \gamma_{00} + \gamma_{20} {\text{Female}}_{\text{ij}} + \gamma_{01} {\text{Knowledge}}_{\text{j}} + \gamma_{02} {\text{GSTPACK}}_{\text{j}} + \gamma_{03} {\text{Years}}_{\text{j}} \\ & + & (\gamma_{10} + \upsilon_{{1{\text{j}}}} ){\text{Pretest}}_{\text{ij}} + \upsilon_{{ 0 {\text{j}}}} + r_{\text{ij}} . \\ \end{aligned} $$

Holding teacher factors constant, γ 00 is the intercept (predicted student outcome) for the male group, γ 20 is the intercept change for the female group, and γ 10 is the pretest slope. γ 01 to γ 03 are the students’ outcome changes due to the effects of the three teacher factors. Because the pretest was taken before the students were taught by the teachers with the Climate Change curriculum, it would not be meaningful to add the three teacher factors for the level-2 slope of the students’ pretest scores as we did for the level-2 intercept. The random effects υ 0j is the departure from the average male intercept, υ 1j is departure from the average pretest slope, both due to teacher variability, and r ij the random error for student i of teacher j.

Results

The pretest and posttest assessment data were organized and sorted to include only those students who had completed both assessments (N = 956). Correct responses were tallied for the items. Paired-sample t test analyses were conducted to compare the pretest and posttest results of the multiple-choice items. Overall results regarding the use of the Climate Change curriculum showed significant improvement in urban middle-level students’ climate change science understandings for the entire assessment and for each topic area subgroup (see Table 2). The large effect sizes were derived from dividing the pre- and posttest mean score difference by the pooled standard deviation (Cohen 1988) for the entire assessment and for each topic subgroup. The standard effect size for the urban students in our study had an effect size of 1.29 for the entire climate change assessment measure. These results speak favorably to using the geospatial curriculum approach to promote growth in students’ climate change understandings.

Table 2 Paired-sample t tests for climate change pre- and posttest achievements: overall and subscale multiple-choice (MC) items (top) and overall open-response items (bottom)
Full size table

Paired t tests for each individual item also revealed significant gains (p < .001) from pretest to posttest for all but one item (item #20 in “Appendix 1”) indicating that students had difficulty distinguishing heat capacity from the albedo effect. Item analysis revealed seven posttest items that were not answered correctly by more than half of the students (items #1, 3, 9, 11, 14, 19, and 20 in “Appendix 1”). After the curriculum implementation, many students still had misunderstandings about the volumetric composition of atmospheric gases, the source of ozone in the lower troposphere, the magnitude of average annual temperature rise in most places during the past 50–60 years, and the timescale differences between weather and climate. In addition, many students had difficulty distinguishing significant greenhouse gases from non-greenhouse gases.

Paired-sample t test analyses were also conducted to compare the pretest and posttest results of the open-ended response items. Students’ scores significantly increased from pretest to posttest (see Table 2) with a medium effect size of 0.54. Overall, the students’ proficiency levels in their responses increased. “Appendix 3” displays the student response summary to the three open-ended items. After the curriculum implementation, more students were able to articulate an increased number of different types of human activities that are causing the long-term increase in carbon dioxide levels over the last 100 years, identify a larger number of behaviors they could implement to reduce or prevent further emissions of carbon dioxide into the atmosphere, and identify a greater number of societal practices to lower the levels of carbon dioxide in the atmosphere.

OLS Regression Analysis on Student Climate Change Posttest Scores

OLS multiple regression analysis using the hierarchical/blockwise method in IBM/SPSS 21 was conducted to investigate whether and how well three teacher factors accounted for the variance in students’ climate change posttest scores, after controlling for students’ gender and pretest scores. The two student variables were entered in Block 1 and the three teacher variables were in Block 2. Following the procedures suggested in Field (2009, Chapter 7), we first checked the data and OLS regression assumptions, with satisfactory findings. Based on the standardized residual and maximal Cook’s Distance, we found no outliers, or extreme cases with undue influences, in the dataset that might bias the prediction. Although two students out of the sample (N = 956) had standardized residual bigger than 3.00 standard deviations (3.19 and 3.04, respectively), the maximal Cook’s distance was only 0.016, being smaller than the cutoff 1.00. Therefore, the two students were kept for the regression analysis (Field 2009). Next, a linear relationship was found between the outcome and each predictor based on the scatterplots and correlation matrix (see Table 3). Finally, there was no multicollinearity issue due to high correlations between the predictors, based on the correlation matrix and the collinearity statistics (tolerance values ranging from .81 to .99, and all variance inflation factor values below 1.24). The explanatory variables had Pearson correlations that ranged from −.14 to .34 with each other and from −.05 to .65 with the outcome (Table 3). Pretest scores were the variable that had the highest correlation with the posttest scores (r = .65).

Table 3 Pearson correlations (one-tailed) among variables predicting the CC posttest scores
Full size table

The results for the regression are presented in Table 4. Student gender and the pretest score in Model 1 significantly explained 42 % of the variance in the climate change posttest scores, R 2 = .42, F (2, 953) = 344.10, p < .001. Pretest score was a significant predictor for the outcome (p < .001), and gender was marginally significant (p = .058). In the full model (Model 2), after controlling for students’ gender and pretest scores, the teacher variables contributed 3 % additional variance in the outcome, R 2 = 0.45, ΔR 2 = .03, F increment (3, 950) = 18.70, p < .001. Climate change pretest scores were still a significant predictor for the outcome (p < .001), and gender was also significant (p = .047). Among the teacher variables, years of teaching was a significant predictor for the outcome (p < .001), GS-TPACK was marginally significant (p = .060), and teacher knowledge in climate change was nonsignificant (p = .92).

Table 4 Hierarchical multiple regression analysis for the Students’ CC posttest scores (N = 956)
Full size table

In Model 2, assuming the values for the pretest score and the three teacher variables all to equal 0, the predicted posttest score for a typical male student was 8.46 out of a possible total of 28.00, whereas being a female student had 0.57 lower points than a male student in the posttest score, p = .047. Pretest scores were the strongest significant predictor of the outcome: controlling for everything else, each-point increase in the climate change pretest score was associated with 0.81-point increase in the posttest score, β = 0.62, p < .001. The pretest scores had a semi-partial correlation at .596, indicating that this variable uniquely explained 36 % (sr2 = .5962 = .36) of the variance in the outcome. The second strongest significant predictor of the outcome was teachers’ years of teaching: controlling for everything else, each-year increase in teaching was associated with 0.25-point increase in the posttest score. However, the squared semi-partial correlation for teachers’ years of teaching uniquely explained only 3 % of the variance in the outcome (sr2 = .172 = .03).

Results from HLM

Table 5 presents the fixed effects estimates and variance–covariance estimates for the two-level random intercept and random pretest slope HLM, predicting the student climate change posttest scores at Level 1.

Table 5 Fixed effects’ estimates (top) and variance–covariance estimates (bottom) for the two-level random intercept and pretest slope HLM predicting students’ CC posttest scores
Full size table

Both student variables (gender and pretest) were significant predictors for student posttest scores, p < .05. Controlling for everything else, the posttest scores were 0.66 point lower for female students than males students, p = .01, and each-point increase on the pretest was associated with 0.68-point increase on the posttest scores. None of the teacher variables (climate change knowledge, GS-TPACK, and teaching years) had significant association with the students’ posttest scores, p > .05. The teacher variation was significant on the average posttest scores (p = .047), but nonsignificant, or marginally significant, for the pretest slope (p = .089). That is to say, the students’ posttest score means across the 12 teachers were significantly different from each other, but the effects of students’ pretest scores on their posttest scores were almost equal (or marginally significantly different) regardless of their teachers. The student-level residual variance decreased by 39 %, from 22.81 in the unconditional model to 13.98 in the random intercept and slope model, which is a substantial decrease due to the student- and teacher-level variables and the two random effect terms in Level 2.

Discussion

The aim of this study was to develop a coherent climate change science curriculum for urban middle-level students using a geospatial curriculum approach, determine its ability to impact climate change science content knowledge gains, and investigate both teacher and student factors that may account for climate change knowledge achievement. The results from this study provide support for the effectiveness of the geospatial curriculum approach to enhance the climate change content knowledge of urban middle-level students. The findings also provide support that a geospatial curriculum approach may be an effective science curriculum approach for learners in urban middle-level education.

The findings in this study build on the work of other researchers who have investigated the implementation of technology-integrated science curriculum using geospatial technologies (Barnett et al. 2012; Bodzin 2011; Edelson 2004; Kulo and Bodzin 2013). Virtual globes such as Google Earth are interactive visualization tools that can be designed to enable learners to manipulate, analyze, and synthesize spatial data in novel ways (Bodzin 2011) and support the development of contextually rich learning environments that promote higher-order thinking skills, meaningful learning, and authentic scientific inquiry (Bodzin 2011; Bodzin and Anastasio 2006). Visualizing the spatial relationships among datasets assists in the cognitive aspect of learning and promotes deeper understanding of content (Sinton and Lund 2007). In the Climate Change curriculum, interactive visual interfaces and tools that are inherent to Google Earth provided students with useful ways to analyze spatial data to investigate the effects of increased anthropogenic climate change. For example, using Google Earth, students are provided with a user-friendly tool set to analyze changes in Arctic sea ice melt data, observe spatiotemporal evidence of declining coral reefs, and analyze predicted effects of sea-level rise in coastal areas on the landscape. These types of geospatial learning activities provide learners with more enhanced learning opportunities to understand certain components of climate change science compared to more typical curriculum approaches of science learning that occur in US urban middle-level classrooms that may rely more on non-technology-integrated approaches to learning.

After receiving instruction with Climate Change, students had a better understanding of how human activities affect climate change. After the curriculum implementation, the students identified more anthropogenic sources that contribute to climate change, environmental impacts, and solutions to enhanced anthropogenic climate change at a personal and societal level. Students identified a variety of actions that they and their families could undertake to reduce their carbon footprint. It is likely that curriculum coherence may have played an important role to promote student learning. Climate Change was designed with learning goals coherence (Schwartz et al. 2008) to help students develop deep understandings about climate change science with a carefully planned interrelated set of conceptual topics and geospatial learning activities based on important climate change science learning goals using a novel curriculum design model. The curriculum focused on developing deep and rich understandings of fundamental climate change science concepts in the areas of the atmosphere, Earth system energy balance, weather and climate, greenhouse gases, paleoclimatology, and environmental impacts of human-induced climate change. Connections among these areas were made explicit through the geospatial and other learning activities.

Previous studies described earlier in this paper have indicated that the teaching and learning of climate change science is conceptually challenging. While significant gain scores were noted for all but one assessment item, many students still had difficulty understanding certain climate change concepts. A four-week curriculum may not be enough time to devote to the teaching and learning of climate change science with urban middle-level learners to enable deep meaningful understandings for all concepts pertaining to atmospheric composition, timescale differences between weather and climate, albedo effect, and greenhouse gases. More explicit instruction beyond what was implemented with the Climate Change curriculum in the classroom may be needed to enhance learner understandings of these topics.

The second research question investigated factors related to both students and teachers that may account for students’ climate change knowledge achievement. The results from the OLS multiple regression and the two-level HLM both found the student climate change pretest scores to be a significant factor for the posttest scores (p < .05) and gender at least marginally significant (p < .10). However, years of teaching experience was significant in the OLS regression (p < .05) but nonsignificant in the HLM (p > .05). As revealed in the HLM, the students had significantly different posttest score means across the 12 teachers, but almost equal (or marginally different) slopes (effects) for their pretest scores across the teachers. The almost uniform effects for the pretest among the teachers made sense in terms of the temporal design of this study: the pretest was taken before the students were taught by the teachers with the Climate Change curriculum.

It is worth noting that the significant posttest mean difference across the teachers might have resulted from the students’ academic class tracking differences among teachers or from school variable differences (e.g., the school percentage of students with free lunch). However, we ignored the class tracking in this study because two of the schools included classrooms with heterogeneous student populations due to mixed academic tracking levels. The school variation was also ignored because we had 4 non-randomly selected schools only.

From the methodology perspective, when the ICC (47 %) from the unconditional HLM indicates a substantial proportion of variance in the student outcome is between teachers, using the OLS regression method would be inappropriate for ignoring the teacher variation in the nesting data structure. To assign the higher-level teacher characteristics on the three teacher variables to all students within each teacher would lead to non-independence of observations because all students within a particular teacher would have identical data on each teacher variable. Meanwhile, we were aware of the fact that we had a small and non-randomly selected sample of 12 teachers in total, so the HLM may be underpowered for fixed or random effects at the teacher level. In the future studies with a much larger sample of classroom teachers, it is possible that the fixed effects for the teacher variables and the random teacher effects will be significant.

In the USA, the next-generation science standards (Achieve Inc. 2013) have recently been released to help guide states to develop a coherent science education curriculum across the grade levels. The content areas of Earth’s systems, weather and climate, and human impacts are included as core standard areas for the middle-level science curriculum and include climate change science concepts. The findings from this study illustrate that implementing a coherent middle-level climate change science curriculum using a geospatial curriculum approach to learning can enhance urban middle-level students’ understandings of important climate science topics and help foster a deeper understanding of climate change science. We encourage further studies that investigate how other curriculum approaches to learning with geospatial technologies may be used to enhance the teaching and learning of climate change science with middle-level students.