Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Curriculum materials are an important educational input that has received limited attention in education policy research. High-quality quantitative research (e.g., Agodini et al. 2010; Bhatt and Koedel 2012; Koedel at el. 2017) suggests that choice of curriculum materials may have substantial impacts on student achievement (a tenth to a fifth of a standard deviation in most studies). However, it is difficult to identify the true effects of curriculum materials on student outcomes because of limited data available to conduct high-quality quantitative studies.

The selection, implementation , and effects of curriculum materials are important issues for scholarly research because high-quality curriculum materials can potentially produce measurable differences in student achievement. For school and district leaders, curriculum materials are likely less controversial and lower in cost than human resource interventions—such as staffing changes or performance evaluation . Curriculum interventions also have the potential for very high benefit/cost ratios, given the low marginal cost between curriculum options (Chingos and Whitehurst 2012).

We begin with a review of relevant literature on the adoption, use, and effects of curriculum materials. Throughout this chapter, we use the term “textbook” to refer to the traditional paper volume that contains a comprehensive subject-specific curriculum—textbooks have been the focus of virtually all of the research we discuss below. “Curriculum materials” includes textbooks, as well as the ever-expanding landscape of resources that teachers call upon to implement their lessons. These materials include supplemental reading materials, open online educational resources (e.g., EngageNY), videos (e.g., Khan Academy), diagnostic materials, software (e.g., Google Apps for Education), and websites where teachers share lessons and resources (e.g., Pinterest, TeachersPayTeachers). Importantly, much of the research on textbooks was published before the widespread availability of these supplemental materials that are largely disseminated online. Therefore, we see the study of curriculum materials as being particularly relevant for future scholars seeking to understand the changing nature of classroom resources available to teachers.

We describe some methods that can be used for researching the adoption, implementation , and effects of curriculum materials, as well as the challenges associated with each method. We describe quantitative methods such as propensity score models, value-added models, and multilevel modeling for estimating achievement effects. We also describe how qualitative data can supplement quantitative data and provide a more complete understanding about the use of curriculum materials. These methods can be used to investigate a variety of policy-related questions about the adoption of curriculum materials, alignment of the classroom curriculum to the intended curriculum of the standards, and fidelity of implementation of curriculum materials. We suggest areas for future research based on the data that currently exist as well as the changing nature of the materials themselves.

Review of Relevant Literature

Opportunity to Learn

Much of the research on curricular materials is grounded in the opportunity to learn literature (see McDonnell 1995). Carroll (1963) introduced the concept of opportunity to learn (OTL) as one of several factors that affect student outcomes, along with aptitude, perseverance, and quality of instruction. Students’ opportunity to learn a given topic is determined by teacher- and school-level decisions. Factors contributing to OTL include the length and content of classes, the order in which lessons are taught, the amount of time given to teach a specific topic, and the content to which students are exposed through curriculum materials (Kurz 2011).

Curriculum materials affect student learning through the content included in the materials (e.g., whether and how two textbooks differ in their coverage of multiplying fractions (Polikoff 2015)) and through teacher implementation of the materials (e.g., how teachers choose to implement the fraction multiplication lessons of the books (Remillard 2005)). OTL has been used as a central argument in court cases ruling that students from low socioeconomic backgrounds should have access to the same quality of curriculum materials as their higher-SES peers (e.g., Eliezer Williams et al., v. the State of California et al., 2000). In these court decisions, curriculum materials have been cited as a key policy lever associated with student learning.

Opportunity to learn can be defined and measured across multiple levels of curriculum. Some research (e.g., Porter and Smithson 2001) differentiates between the intended curriculum and the enacted curriculum. The intended curriculum refers to the skills that students are expected to know as determined by factors such as state standards, district pacing guidelines, or the content of a teacher’s guide. The enacted curriculum refers to the curriculum that is actually delivered to students in class via instructional materials and teaching methods. Under the OTL framework , curriculum materials contribute to student learning opportunities as they form a bridge between the intended and the enacted curriculum (Kurz 2011). For example, the Common Core State Standards (or a closely-related or renamed version) represent the current intended curriculum in many states and establish the content and skills that students must master in each grade. Publishers then produce new materials that are aligned to these standards and that help teachers interpret the standards and determine how they deliver instruction.

The Adoption of Curriculum Materials

Curriculum materials can be adopted at the state, district, school, or even classroom level, depending on state and local laws. This means that to study curriculum selection, implementation , and effects, researchers need to have an understanding of the type of adoption practiced in the region being studied. Roughly half of the states publish a list of approved textbooks evaluated by the state department of education and determined to meet the needs and standards of the state. In some states, such as Mississippi, districts are required to adopt from the approved list of materials, while in states such as California and Texas, districts are encouraged to adopt materials from the list but are also free to make local decisions about curriculum material adoptions.

Previous research on the adoption of curriculum materials focused on the state-level processes related to evaluation and selection of textbooks (e.g., Finn and Ravitch 2004; Stein et al. 2001). Relatively little research exists on the adoption decisions made at the district or school level, especially if we only consider research from the standards era. The most comprehensive study (Zeringue et al. 2010) identified several factors that matter to district leaders when making adoption decisions, including perceived teacher support , evidence of quality of materials, and resources available for purchase and implementation . We recently conducted interviews with district leaders in California and found these factors to be important in district-level decision-making, with teacher voice being an especially influential factor. Other factors that seem to matter include the support of the county office of education, the cost of materials distributed over the length of the adoption cycle, and district characteristics such as the percentage of English language learners and quality of technology. Our interviews also revealed that the previous literature on textbook adoptions needs to be revisited as districts consider options such as digital or hybrid curricula and open educational resources available online.

The Implementation of Curriculum Materials

Curriculum materials offer teachers suggestions regarding the sequence and pacing of lessons, the scope of the subject matter covered in a year, and the strategies to be used for instruction. The choices that teachers make regarding the implementation of the lessons shape the enacted curriculum—the curriculum that students actually experience in the classroom. In short, textbooks and other curriculum materials are an important factor in the relationship between the intended and the enacted curriculum, and therefore are a contributing factor to differences in student opportunity to learn (Schmidt et al. 2001).

Not every teacher uses traditional textbooks as their primary source of instruction, but it is clear that teachers do consider these materials in shaping their lessons. A recent national sample of teachers from the American Teacher Panel indicated that, while they use a variety of sources to plan and implement lessons, most teachers still use traditional textbooks for at least some, if not all, of their planning (Opfer et al. 2016). From this survey, we gain two important insights regarding teacher use of materials during the Common Core era. The first is that textbooks remain an influential factor in teacher planning and are thus a relevant policy lever. The second is that there is a proliferation of non-traditional materials that should be researched more extensively. These non-traditional materials include open online educational resources, interactive or hybrid textbooks (with features online and in print), and websites where teachers share materials with other teachers (either for free or for a charge).

Because textbooks remain widely used by teachers, they represent an important policy lever that can be used to improve student achievement. That is, improving the quality of textbooks or the processes by which textbooks get into teachers’ hands could materially improve instruction and student learning due to the ubiquity of these materials. The textbook alone, however, cannot ensure high-quality instruction, as the teacher must interpret, design, and implement lessons. This process of teacher participation creates variations in resource implementation (Remillard 2005). Such variations occur as a teacher may implement the lessons with complete fidelity, may use the textbook as one of many available resources, or may interpret and interact with the text in a co-constructive relationship (for a review of teacher use of curriculum materials, see Remillard 2005). The relationship between teacher and curriculum materials is one that requires further research, especially in the context of the emergence of open online resources and other technological advances. Teacher implementation of materials is an important variable for researchers studying the effects of textbooks on student outcomes.

The Impact of Curriculum Materials

In spite of the substantial discretion teachers have when implementing materials, recent rigorous quantitative evidence suggests that simply adopting one book over another can produce meaningful effects on student achievement. One recent large-scale study (Agodini et al. 2010) randomly assigned elementary mathematics textbooks to schools, finding statistically significant differences in their effects on student learning. These differences were 0.10 to 0.15 standard deviations in magnitude, enough to move students from the 50th to the 57th percentile, depending on grade. Three recent studies use matching methods (which we describe below) to investigate the impact of elementary mathematics textbooks in California, Florida, and Indiana; again, each study found that at least one textbook outperformed the others (Bhatt and Koedel 2012; Bhatt et al. 2013; Koedel et al. 2017). Together , these four recent studies provide compelling evidence that textbooks matter for student learning.

Applications to the Study of Leadership and Policy

To date, there has been little comprehensive analysis of widespread patterns and effects of textbook usage due to a dearth of available data (Chingos and Whitehurst 2012). As mentioned above, there are three recent matching studies, but these are all in elementary mathematics, and they are only in three states. Clearly, there is a need for research in other subjects, other grades, and other states. Table 10.1 presents some suggested research questions and data sources aligned to the three principal areas of research related to curriculum materials: adoption, implementation , and effects of curriculum materials.

Table 10.1 Illustrative research questions and data sources related to the study of curriculum materials adoptions
Full size table

There are at least three main topics that merit continued scholarly investigation. First, regarding adoption, qualitative analysis of how adoption decisions are made at a district level can illuminate the factors prioritized in the selection process. Such insights could help researchers and policymakers to provide evidence to districts that might improve their selection decisions. Second, regarding implementation , qualitative or survey analysis of teacher implementation can provide insight into how the enacted curriculum may differ from the intended curriculum, which could enhance our understandings of students’ access to the curriculum. Third, regarding impacts, collection, and analysis of textbook adoption data at the school or district level (i.e., cataloging which books are purchased in which districts and schools across entire states) can inform policymakers about which resources are associated with higher student achievement and illuminate how access to high-quality materials may differ between classes, schools, and districts. Ultimately, understanding how textbooks are adopted and implemented , and the impact of these decisions on student achievement outcomes can create a more nuanced understanding of student learning opportunities, as outlined under the OTL framework .

Analyzing Textbook Adoption Decisions

An understanding of textbook adoption decision-making processes is important for situating studies of implementation and effects. For example, one might expect to find more variation in student outcomes in a school district where principals or teachers are allowed to make their own decisions about adopted materials than in a uniform-adopting district. Adoption decisions can take place at the state, district, school, or even classroom levels, so it is essential for researchers to first identify the appropriate level in the area under study. This information can often be found on state or school district websites, and it can be confirmed through phone calls or emails.

In terms of the adoption processes themselves, researchers can qualitatively investigate the nuances of adoption processes through interviews with district leaders, teachers, principals, members of the state department of education, or any other parties involved in the selection of curriculum materials. A straightforward qualitative analysis of evaluation criteria and adoption decisions can provide comparative data across districts.

Additional scholarship might study textbook adoption decisions through ethnographic methods, such as participating in and observing textbook adoption committee meetings. A staple of ethnographic work is what Geertz (1973) referred to as “thick description,” which aims to uncover not only what happens but also how relevant actors and observers interpret what happens. Studying textbook adoption decisions through an ethnographic lens could thus provide insights into how individuals make meaning of curriculum materials, including how actors perceive their own curricular needs, how they evaluate materials based on these needs, and how they use these materials accordingly. Ethnography can also shed light on how institutional structures and cultures dictate curriculum evaluation and usage within the classroom. Ultimately, a deep and nuanced analysis of how decisions about curriculum materials are made would allow researchers and policymakers to provide targeted assistance to aid in this process.

Studying the Implementation of Textbooks

There are two main questions of interest under the broad topic of textbook implementation. First, how do teachers actually use the materials that are adopted, and how do they supplement these materials? Second, what is the content of these materials, and how does it affect student opportunity to learn?

Analyzing Teacher Use of Materials

We propose studying teacher use of materials through both qualitative case study research and large-scale survey research. Case studies provide extensive and holistic descriptions of a singular unit (Merriam 1998) and allow researchers to explore how occurrences in this unit are influenced by context (Stake 2005). Thus, studying teacher use of materials through case study would allow researchers to explore how teachers are utilizing the materials they are given and how their pedagogical decisions are influenced by the political, social, and organizational culture within their school as well as their prior training and beliefs. For example, the degree to which teachers have autonomy over their classroom, their access to common planning, and their sharing of resources could profoundly impact their use of textbooks within the classroom. Understanding these social and organizational structures within districts, schools, and departments and how they impact teacher pedagogy requires in-depth analysis and experiential knowledge, which are well suited for a case study approach.

A limitation of case study research is its generalizability. Thus, to explore how common practices are in other schools, districts, and states, we propose using large-scale survey research. We have begun to explore these issues with state-representative samples of teachers, asking them detailed survey questions about their use of curriculum materials (see Kane et al. 2016, for an earlier version of this work). We will use these survey responses both to describe curriculum use at scale and to construct predictors to explore variation in textbook effects on student outcomes.

Additional research might focus how materials are used in states, districts, and schools from an equity perspective. For example, how does the use of curriculum materials differ within schools based on a student’s prior achievement or status as an English Language Learner (ELL) or Special Education (SPED) student? Topics of consideration might include whether remedial students are equally likely to be exposed to materials that require high cognitive demands and whether ELL and SPED students are being provided materials that are appropriate for their learning needs. Ultimately, whether or not students with different background characteristics or special designations have access to high-quality and academically appropriate materials is an important policy question that we can begin to address by analyzing the implementation of the materials by teachers and schools.

Analyzing the Content of Textbooks

In order to gauge how textbook adoption choices influence students’ opportunity to learn, we can analyze the content of the materials. Using the Surveys of Enacted Curriculum (SEC) framework (Porter 2002), we can quantitatively code entire textbooks, creating an index of how thoroughly the book covers standard topics, and the level of rigor with which each topic is addressed (for an example, see Polikoff 2015). Variants on the SEC methods have been in use for over 20 years (see Porter 2002 for a history; for other examples see Porter et al. 2007; Polikoff 2012).

There are existing SEC frameworks in mathematics and English language arts that have recently been revised to study implementation of Common Core and other content standards (see Porter et al. (2011) for an analysis of the Common Core using the SEC and www.c-sail.org for recent work to update the SEC languages). These define content at the intersection of specific topics and levels of cognitive demand. Independent coders use the frameworks to assign topics and levels of cognitive demand to sections of text. The results are then averaged across raters to arrive at a complete representation of the content in the textbook or other curriculum material.

Once the coding is complete, the data can be used to calculate alignment indices or other descriptive measures of textbook content. For example, we can calculate the alignment of a book with a set of content standards, indicating the book’s overall coverage of the topics and cognitive demands emphasized in the standards. We can also report the proportion of each book’s content on focal content strands or at lower and higher levels of cognitive demand. Finally, we can compare alignment and other descriptive indices across textbooks. To date, only mathematics materials have been studied using an SEC content analysis. It is possible that science, ELA, or history/social studies materials could be coded, though these are somewhat more complex than mathematics due to the nature of content in the subjects.

Recent research demonstrates that the content analysis of mathematics textbooks (Polikoff 2015) recommends specific strategies for simplifying the content analysis procedures (Polikoff et al. 2015). These papers use the SEC framework to measure the alignment of several popular math textbooks to the Common Core State Standards for math, on the principle that better aligned materials offer students a better opportunity to learn the standards. This work found that even the most popular textbooks claiming alignment to the Common Core math standards were not well aligned, particularly with regard to the cognitive demand required of the standards. Given the role of textbooks in influencing teachers’ instruction, especially during the early years of a standards transition, these kinds of content analyses can shed important light on likely areas of alignment and misalignment in teachers’ instruction. Furthermore, they can point the way toward areas of needed supplementation. Finally, these measures of alignment or content coverage can provide a measure of the relative quality of textbooks—at least with regard to their coverage of the standards—that teachers and district leaders may wish to use when making adoption decisions.

Studying the Impact of Curriculum Materials

Probably the question of greatest interest to policymakers is the impact of textbook choices on student learning. Here, we briefly outline methods for (a) gathering the necessary data and (b) conducting the actual impact analysis.

Collecting Data on Textbook Adoption Patterns

The first step in identifying the impact of curriculum materials is collecting the necessary data to conduct a secondary data analysis. The preferred approach will vary based on the state and what data are available, as most states do not make textbook adoption data available in any form. California is among just a handful of states that do provide publicly available information on schools’ adopted textbooks. Every school in the state is required to publish a yearly School Accountability Report Card (SARC) that includes information about the quality and availability of textbooks (typically this means that titles and adoption years are provided). For our work (Koedel et al. 2017), we have manually downloaded and recorded the SARC information for every school serving elementary and middle grades (n ~ 7600) for the years 2012–13 to 2015–16. This is a time-consuming process because there is no standardized format for schools to use when entering textbook information. The challenges of this process are described in detail elsewhere (Koedel et al. 2017).

A second option is to use state-level purchase records in the states that keep them. While we know of no definitive list of such states, we are aware that Louisiana, New Mexico, Tennessee, and Texas all track this information. For instance, the Texas Education Agency (the state department of education) holds records on all curricular materials purchased at each district site, and the data are updated daily. These data are recorded at the district level and are the most comprehensive records of curriculum materials used because they include everything from traditional textbooks to online supplemental programs to novels—anything that districts use money to purchase. Similar data are also available in New Mexico and Tennessee, and some other states we are not aware of may also track purchase data in this way.

A third option is to collect the data by contacting individual school districts. Surveys can be used, but these require incentives to obtain even moderate response rates. In our work, after attempting and failing at population surveys in Illinois, New York, and Florida, we found that districts have responded to a request for information filed under state Freedom of Information Acts/Laws (FOIA/FOIL). State-specific templates for FOIA letters are readily available online and can be mailed to school or district offices. This method is likely to yield a high response rate, as districts are required to provide any existing records containing the requested information (though they are permitted by law to charge for expenses associated with fulfilling the request, very few districts do this). However, even with clear instructions, there is a great deal of variability in the quality and completeness of information provided by individual schools and districts. The FOIA method could be used in states where data on purchases and adoptions are not readily available any other way, but it should be used sparingly as it is seen as intrusive and confrontational. The method is also time consuming because it requires careful tracking of contacts, and the information reported is not in any standardized format. All of these methods of data collection share the burden of being time consuming, though they have the potential to provide a nearly complete picture of the formal textbooks of record in schools and districts in a state.

Analyzing the Impact of Textbook Adoptions on Student Achieve-ment

Once the data on textbook adoptions have been collected, there are multiple analytic options that can be used to attempt to identify the causal effects. For a question such as “Which of the most common elementary school mathematics textbooks has the most positive effect on student achievement?” the kinds of matching methodologies used in Koedel et al. (2017) are appropriate. This research uses propensity score models and longitudinal school-level test data to match schools on a variety of demographic and achievement variables thought to be related to textbook adoption decisions (e.g., school-level demographics, poverty, and geographic variables) and track subsequent achievement trends. Koedel and colleagues’ most recent paper uses three matching techniques—kernel matching, restricted ordinary least squares, and remnant-based residualized matching (the methods are described in great detail in the paper). If evidence can be provided that the key assumption of conditional independence (that there are no unobserved variables related to both the textbook adoption decision and student achievement) is met, these methods can produce causal estimates of textbooks on student outcomes. These conditional independence assumptions are generally explored by first demonstrating balance among the treatment groups on all available covariates and then by conducting falsification tests such as testing for math textbook “effects” in other subjects and looking for effects in years in which they should not exist. All of these methods are described clearly in Koedel et al. (2017), and they have also been applied in two other studies (Bhatt and Koedel 2012; Bhatt et al. 2013). These methods could also be used to investigate heterogeneous effects across student subgroups—Koedel et al. (2017) demonstrates this for student socioeconomic status.

Similar data could also be used to conduct something like a difference-in-differences or comparative interrupted time series analysis (see Murnane and Willet (2011) for a discussion of the DD and CITS methodologies), though we know of no instances where this has been done.

Another approach involving secondary data uses student-level data to conduct value-added analyses at either the school or teacher levels. For example, Kane et al. (2016) calculated value-added models to estimate the impacts of individual teachers on student achievement (any standard value-added model or student growth percentile could be used (see Koedel et al. (2015) for an overview of value-added models). Then, they related these value-added estimates to a variety of measures of curriculum materials used and curriculum implementation indicators. If these methods are similarly paired with efforts that control for pre-adoption differences in schools, such as by controlling for pre-adoption value-added, they can also identify causal impacts of curriculum materials.

An alternative approach to examining the impacts of textbooks with secondary data uses random assignment to generate unbiased causal effects . Agodini et al. (2010) recruited schools from geographically diverse regions of the country and randomly assigned them to investigate the impact of four of the most widely used math textbooks; they found significant achievement differences among the examined books. There are a number of other random assignment curriculum studies listed in the What Works Clearinghouse, but these studies suffer from many problems that substantially limit their utility. For instance, (a) the vast majority of them pre-date recent standards adoptions, (b) the control condition in many of the studies is underdescribed, (c) many of the studies are very small (just a few schools or classrooms), and (d) large proportions of the studies focus on small-scale curricula rather than core/basal curricula. Finally, there may be a difference between schools that choose to adopt a curriculum and schools that participate in random assignment studies that may limit the external validity of random assignment studies.

One complication from the existing impact research is that Agodini et al. (2010) and Bhatt and Koedel (2012) found student achievement effects associated with the same textbook, but one found a positive and the other found a negative effect. While this may be seen as problematic (certainly it is problematic from a policy interpretation standpoint), these divergent findings do not imply any problem with methodologies. Rather, these differences may be attributable to differences in the outcome measures used (different state or study-administered tests) or samples (the Agodini et al. sample was more disadvantaged than the statewide sample in Bhatt and Koedel ). Further research is necessary to determine how curriculum effectiveness may vary across different populations of students and outcome measures.

Recommendations for Novice and Emerging Scholars

Curriculum materials are a relevant topic for scholars of leadership and policy focused on instruction, given the primary role of these materials in shaping instruction. Thus, even if curriculum materials are not a primary area of a scholar’s research, collecting data on the types of materials used, and the manner in which they are used, can be an important contribution. This section offers two broad sets of recommendations for scholars. First, we note the complications that have emerged as textbook markets have changed alongside teachers’ use of supplementary and open source materials. Second, we offer thoughts about the most appropriate sets of methods to use, together and independently, to address important questions about the adoption, usage, and effects of curriculum materials. Overall, our work supports previous scholarship in arguing that it is essential to study the processes of policy implementation (in this case, standards implementation ) at multiple levels in order to see meaningful improvements in teachers’ practice (Knapp 1997).

If trends away from traditional textbooks continue, it will become increasingly important to study what resources are available to teachers, how leaders help teachers navigate these materials, and how these resources are shaping teachers’ instructional practices. The methods proposed here give us a path forward for understanding textbook use and effects, but as materials change, so too will the research needs. The availability of resources means that teachers are supplementing the traditionally adopted curriculum with materials they believe are specifically suited to their students’ needs. We know little about widespread use of curriculum materials to begin with, and the degree of variation between classes is presumably affected by the expanding market for supplemental materials. We expect that future research in the area of curriculum materials can shed insight into the ways in which advances in technology are affecting the implementation of both traditional and non-traditional materials. This research can be helpful in creating a fuller picture of the enacted curriculum, including its alignment to the intended curriculum and its variation between classes and schools. However, these questions will be difficult to answer due to the even greater difficulties associated with trying to learn what teachers are doing on a day-to-day basis.

To gain a more comprehensive perspective of teaching and implementation , we recommend using both quantitative and qualitative measures of classroom instruction. Prior research has quantified classroom instruction using measures such as teacher logs (Rowan et al. 2004), and an analysis of the types of activities in which students were engaged in the classroom (Tarr et al. 2013). Any research tool that allows researchers to go into the classroom and analyze the content, duration, and quality of lessons would be helpful to the body of research on curriculum materials. It would also be useful to talk to teachers and understand why they are supplementing their traditional textbooks, how they find and evaluate supplemental materials (including the role of school leaders), and what effects supplementation has on the coherence and quality of their instruction. These questions are best answered qualitatively through rigorous case study methods.

There is also work to be done on the study of computer-based curriculum materials, about which we know little. Who adopts these materials, how do they differ from traditional textbooks, and what are their effects on instruction and learning? Which websites are the most popular, and how widely used are they? Are the materials on some sites better than others? Even within a website, there may be large variations in the quality and alignment of lessons—do these variations relate to teachers’ likelihood of selecting the materials? Furthermore, there is the simple issue that it is no longer the case that traditional textbooks are “necessary,” since teachers or districts could pull together curricula from online sources. Would this be a good idea? Are districts that use created “units of study” better off? Would moving away from textbooks be a cost-effective solution? And do all schools have the infrastructure necessary to support these trends?

There are other aspects related to textbooks that also merit additional research. Pacing guides and other resources are frequently provided to teachers by schools and districts, but we know little about these materials. They can be analyzed using some of the same methods that we use to study curriculum materials. For example, content analysis could be used to measure alignment of district-provided pacing and implementation materials to state standards or to teachers’ enacted curriculum. Again, pacing guides represent another mediating variable separating the standards policy from teachers’ actual instruction.

Another important topic is the process by which textbooks get into schools. Researchers could add to the body of literature on materials adoption by looking at the processes by which schools and districts adopt materials and how these differ by key types of schools (e.g., charter schools and magnet schools). This is likely a key leverage point for getting better materials in the hands of teachers, but we know very little about how these decisions are made.

A final recommendation is to study the role of instructional leadership in helping teachers implement curriculum materials. Teachers likely need support to implement new materials, and leaders undoubtedly play a role in bringing effective professional learning opportunities to teachers. Leaders also play a key role in establishing a coherent instruction vision within a school or district. These recommendations are just a few starting points that researchers could pursue to enhance the scholarly literature on curriculum materials.

Chapter Summary

Drawing on the OTL literature, we argue that curriculum materials are an important educational input that affects student learning. Yet, despite the importance of this work, research on the adoption, implementation , and effects of supplementary curriculum materials has been relatively minimal. In this chapter, we review the relevant literature, discuss potential topics and methods for future scholarship, and address the policy implications with respect to each of these categories.

Curriculum materials shape student learning opportunities by creating a bridge between the intended and enacted curricula, ultimately affecting the content to which a student is exposed. In reviewing existing literature on the adoption, implementation , and effects of these materials, we underscore three key findings: (1) districts generally have similar adoption processes for selecting textbooks, but this is based on very limited evidence; (2) textbooks remain widely used by teachers, but they are increasingly being supplemented by additional materials and are implemented with varying degrees of fidelity; and (3) some books have larger effects on student achievement than others, as demonstrated through rigorous quantitative analysis.

We propose that additional scholarship is necessary to understand how textbooks are being adopted, used, and supplemented in the classroom and how these choices impact student learning. Correspondingly, we suggest: (1) qualitative analysis of textbook adoption decisions through methods such as interviews and ethnographic studies at the school and district level; (2) qualitative analysis of the utilization, supplementation, and content of curriculum materials through case study and survey methods; and (3) quantitative analysis of the effects of curriculum materials on student achievement, using matching, value-added, or experimental methods. Ultimately, we argue that curriculum materials have profound impacts on teacher practice and student exposure to content. As such, they are an important educational input that warrants further consideration by researchers and policymakers alike.

Recommended Readings

Bhatt R., & Koedel, C. (2012). Large-scale evaluations of curricular effectiveness: The case of elementary mathematics in Indiana. Educational Evaluation and Policy Analysis, 34(4), 391–412.

Bhatt and Koedel use propensity score methods to estimate the effects of elementary mathematics textbooks on student achievement in Indiana. This paper is the source of the propensity score methods described in the paper and offers a guide for researchers who might want to replicate the work in other states, subjects, or grades.

Polikoff, M. S. (2015). How well aligned are textbooks to the Common Core Standards in mathematics? American Educational Research Journal, 52(6), 1185–1211.

Polikoff provides an introduction to and description of content analysis methods for analyzing the alignment of textbooks to content standards. The methods can also be used to describe and compare the content among several sets of curriculum materials.

Remillard, J. T. (2005). Key concepts in research on teachers’ use of mathematics curricula. Review of Educational Research, 75(2), 211–246.

This review summarizes what is known about teachers’ use of curriculum materials in mathematics. While the work pre-dates recent moves toward online and supplementary materials, the review has great relevance for understanding the relationship of curriculum materials with the enacted curriculum.