Mathematics education reform has been tightly linked with teachers’ professional development. Indeed, professional development is widely believed to be at the center of reform (e.g., Borko, Koellner, & Jacobs, 2014) while evidence is growing that quality professional development leads to gains in student learning (Desimone, 2009; Meiers & Ingvarson, 2005). However, professional development for teachers is often conducted in person and in small groups—hence costly and localized. If, indeed, we view professional development as necessary for the reform of mathematics instruction, it is imperative to search for approaches and tools to design scalable and sustainable professional development that would benefit larger numbers of teachers and communities, while maintaining fidelity to its core principles and impact. Our efforts reflect a general interest in mathematics education to have an impact on practice in large and measurable scale (e.g., Cai et al., 2017). Hence, the purpose of the current study was to explore different ways for teachers to engage in professional development and how these approaches might enable the field to scale up these efforts in a sustainable manner. We use early algebra as the context of our work.

FormalPara Professional learning experience

Professional development is a term inclusive of a broad range of activities. Little (1987, p. 491) describes it as “any activity that is intended partly or primarily to prepare paid staff members for improved performance in present or future roles in the school districts”. Kennedy (2016) notes that in these broad terms, professional development is nearly ubiquitous among teachers and the vast majority of these experiences take the form of face-to-face workshops. However, workshops can vary widely in their focus, ranging from general education issues to specific, targeted focus on content. As Kennedy concludes, not all formats are equally effective—in fact, some widely used formats may adversely affect student outcomes.

In recent years, new prospects have emerged for professional development. As a start, advances in digital technology create unique possibilities for designing experiences for teacher professional development that can be integrated widely in courses, or that can be used by teachers, individually, in study groups, and in local or distributed professional learning communities. The availability of the internet allows for online PD in expanded ways. Dede, Ketelhut, Whitehouse, Breit, and McCloskey (2009) highlight several advantages of online PD, including the possibility of accommodating different schedules and reaching geographically isolated teachers. However, concerns are also raised on what might be lost with this professional learning experience (PLE) format, particularly in terms of building trust and local collegiality, or providing teachers with hands-on experiences. Recent research (Fishman et al., 2013; Blanchard, LePrevost, Tolin, & Gutierrez, 2016; Hill, Beisiegel, & Jacob, 2013) has shown that technology-based PLEs can be as effective as in-person ones. However, other studies (e.g., Kennedy, Rodgers, Romig, Lloyd, & Brownell, 2017; Moon, Passmore, Reiser, & Michaels, 2014) suggest that online professional development is not monolithic; the effectiveness of online professional development can vary as the features and focus of the activities shift.

A third PLE approach is the use of educative curriculum materials (ECMs). These materials, as described by Davis and Krajcik (2005, p. 3), are designed to “increase teachers’ knowledge in specific instances of instructional decision making, but also help them develop more general knowledge that they can apply flexibly in new situations”. These characteristics distinguish them from typical teacher guides that offer supports to promote teacher actions, but not necessarily teacher learning (e.g., typical guides may offer solutions to problems, or additional problems). Researchers have identified several areas in which ECMs can support teachers by providing: (1) anticipated student responses and misconceptions, as well as ideas for how to address those responses; (2) information to develop content knowledge, including how mathematics ideas connect across lessons and units; (3) information about the thinking underlying the design of the materials, including rationales for the inclusion of particular ideas, selection of tasks, and sequencing of tasks; and (4) supports for making decisions about instruction, including how to adapt a task or lesson (Beyer, Delgado, Davis, & Krajcik, 2009; Stylianides, 2008).

Several scholars have taken a critical look at these materials. Krajcik and Delen (2017) raised the question of the effectiveness of these materials as a PLE. While some studies (Arias, Smith, Davis, Marino, & Palincsar, 2017) demonstrated that ECMs can support teachers in developing pedagogical content knowledge, others have found that ECMs are most effective when used in conjunction with other types of PLEs (e.g., Dajani, 2017; Donna & Hick, 2017; Pringle, Mesa, & Hayes, 2017).

This growth in PLE complexity brought a renewed interest in identifying efficient PLE formats. This study, done as part of project DELTA,Footnote 2 sought to move beyond the pair-wise comparison of PLE formats, and to compare the impact of three PLE formats on the knowledge and practices of elementary mathematics teachers while holding constant the specific mathematical concepts to be learned and the lessons to be taught. The three formats were (a) attending a facilitated week-long workshop supporting the use of an instructional unit; (b) participating in a self-guided multimedia course supporting the use of the unit; and (c) using the educative resources provided within the unit materials. In this study, the curriculum units used were focused on early algebra. Constraining the study around the topic of early algebra using specific curriculum materials facilitated comparison across PLE formats. At the same time, we cannot claim that the study's findings extend beyond this targeted content and context. The research questions guiding this work include: What is the impact of a scaled-up, practice-based, and research-informed professional development, as experienced in three different formats, on:

  • Teachers’ preparedness for teaching the targeted mathematics?

  • Teachers’ content knowledge of the targeted mathematics?

  • Teachers’ knowledge and use of instructional practices for teaching the targeted mathematics?

1 Theoretical background—rationale

Mathematical knowledge for teaching

Teachers’ mathematics content knowledge plays an integral role in the quality of mathematics instruction students experience in the classroom. This knowledge, known as mathematical knowledge for teaching (MKT), stems from Shulman’s (1986) characterization of teachers’ “content knowledge.” MKT includes a core understanding of the discipline and pedagogical content knowledge and skills related to the demands of teaching, such as making instructional decisions based on knowing how students think about and learn mathematics (Ball & Bass, 2000; Ball, Thames, & Phelps, 2008). Findings from numerous studies have shown that MKT influences teachers’ beliefs and practices—e.g., their ability to focus on the underlying mathematics concepts, to pose and approach rigorous mathematical questions, and to help students make important mathematical connections among key ideas—and is related to student achievement (Hill, Rowan, & Ball, 2005).

Effective professional development

Critical for MKT development are the professional learning opportunities offered to teachers. There are a number of different models of professional development in which teachers can engage. However, workshops have been the most prevalent model for delivering professional development (Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009). Data from the 2012 National Survey of Science and Mathematics Education indicate that roughly 90% of K–12 mathematics teachers who attended mathematics-focused professional development between 2009 and 2012 attended a workshop, with much smaller numbers reporting other professional learning activities such as study groups or receiving feedback from a coach or mentor (Banilower et al., 2012). Despite its prevalence, studies have shown that this type of training experience, when sporadic and disconnected, is ineffective and does not lead to changes in teachers’ beliefs and practices and improvements in student achievement (Darling-Hammond et al., 2009; Guskey & Sparks, 2002; Yoon, Duncan, Lee, Scarloss, & Shapley, 2007). Further, most professional development workshops are conducted in a localized manner; they are costly and can be challenging to scale up to attend to the needs of large school districts.

Over the years, a consensus has emerged within the field about what constitutes high-quality professional development. A considerable body of literature, with some empirical support, has outlined guiding principles for designing and implementing effective professional development (Elmore, 2002; Garet, Porter, Desimone, Birman, & Yoon, 2001; Loucks-Horsley, Love, Stiles, Mundry, & Hewson, 2003). In this paper, we focus on six of the elements commonly cited in the literatureFootnote 3: (1) duration, (2) content focus, (3) coherence, (4) active/practice-based learning, (5) collective participation, and (6) expert facilitation. Table 1 provides a brief summary of each of these elements.

Table 1 PLE elements
Full size table

Teachers’ beliefs

Research suggests that teachers’ beliefs and values about teaching and learning have an impact on their teaching practices (see reviews by Philipp, 2007; Stipek, Givvin, Salmon, & MacGyvers, 2001; Thompson, 1992). Many American elementary school teachers view mathematics as a static body of knowledge that consists of rules and procedures. Consistent with this conception of mathematics and mathematics learning are teachers that view their role as primarily introducing students to new procedures and providing them with step-by-step instructions (e.g., Stigler & Hiebert, 1997). On the other hand, inquiry-oriented teachers have a more dynamic view of mathematics and actively engage students in the construction of knowledge (e.g., Ball, 1993). One may then infer that influencing teachers’ beliefs through various PLC contexts may be foundational for changing teachers’ classroom practices. Hence, in order to better understand changes in teachers’ practice, it is important that we attend to the nature of teachers’ beliefs about mathematics teaching and learning.

1.1 Description of this study

The present study was designed to investigate different practice-based PLEs focusing on three main objectives: (1) learning early algebra content for teaching, (2) using context when teaching early algebra, and (3) observing students and conferring with them about their work. These objectives were addressed through two algebra units from the Contexts for Learning Mathematics series: Trade, Jumps, and Stops (Fosnot & Lent, 2007) for grades 1–3 (approximately ages 6 to 9) teachers and The California Frog-Jumping Contest (Jacob & Fosnot, 2007) for grades 4–6 (ages 9 to 12) teachers. In particular, we were interested in examining teacher learning outcomes related to knowledge, beliefs, and practices when teachers engaged in three distinct PLE formats that offer different possibilities for scaling up efforts to reach large numbers of teachers.

2 Methodology

Teachers were recruited under the conditions that they both agree to random assignment to one of three PLE formats and commit to teach the assigned instructional unit appropriate for students at their grade level during the subsequent school year. Of the 205 teachers enlisted for the study, approximately half of the teachers taught grades 1–3 and half taught grades 4–6. Within these grade bands, teachers were randomly assigned to one of the three PLE formats. Consequently, the study involved six research groups, one for each PLE format at each grade band.

2.1 The three PLE formats as research conditions

Three formats for mathematics teachers’ professional learning, described below, were explored and compared for this study. Participants in all three formats were provided with a copy of the instructional unit corresponding to their grade levels.

Facilitated workshop

The facilitated workshop PLE consisted of a week-long (30 h) course for one of the two instructional units, conducted during the summer. The workshop facilitators were two project leaders who had co-authored the instructional units and had many years of experience facilitating professional development for teachers. The facilitated workshop participants worked together on activities engaging them in learning mathematics for teaching the instructional units, experiencing pedagogical strategies central to the units. Their discussions addressed both the mathematics content they were learning and the ways in which the pedagogical strategies supported their learning. They also studied and discussed classroom episodes presented in video format, and samples of student work from these episodes. The sessions included opportunities for whole and small group discussions about mathematics content and pedagogy. Although they accessed video and other materials during the workshop via a digital learning environment presented in DVD format, they were not allowed to keep the DVD for individual exploration following the workshop. (To avoid possible contamination of the study, we needed to ensure that workshop participants did not inadvertently engage in the second research condition as well.)

Multimedia course

Participants in this PLE were provided with materials to support 25 h of self-guided professional learning within the digital learning environment. All teachers assigned to the multimedia course group received a copy of the digital learning environment DVD for their grade level with a plan for navigating the digital learning environment to guide their individual coursework. The teachers were asked to complete the tasks on the DVD as outlined in the guide, but at their own pace, during the summer. Many of the tasks comprising the multimedia course were identical to those used in the facilitated workshop but experienced entirely through individual interaction with the digital learning environment. The multimedia course used the same video episodes employed in the facilitated workshop, and participants in the multimedia course were prompted to consider reflection prompts comparable with those employed in the facilitated workshop. Once the teachers completed their work with the materials, they sent all of the files produced through their interaction with the digital learning environment to the research team, and kept the DVD to use as they chose throughout the year.

Curriculum resources

Teachers assigned to the curriculum resources PLE format received a copy of the instructional unit appropriate for the grade levels at which they taught, but no other supports to prepare for teaching the unit. However, the curriculum resources include extended discussions regarding the mathematical content as well as suggested lesson plans, incorporating expected student responses and common misconceptions. Many lesson plans are accompanied by transcribed classroom episodes similar to those presented on the DVD. Teachers were encouraged to consider all of these resources as they studied the units and prepared to teach them.

2.2 Sample

We recruited 205 teachers in total from across the USA and Canada (with one participant from the UK). Of these, 148 completed all or almost all of the study requirements, resulting in 23 to 26 teachers in each of the six research groups whose data could be used in the study (see Table 2). Teaching responsibilities ranged from kindergartenFootnote 4 to sixth grade, although most of the participants taught second, third, fourth, or fifth grade. About two thirds of the teachers had completed an undergraduate major or minor in elementary education, about half held graduate degrees in education or mathematics, and 14% held a credential for teaching elementary mathematics specifically. As a group, the participants were quite experienced, averaging 14 years in the field.

Table 2 Participants by PLE format and grade level
Full size table

2.3 Quantitative measures

All teachers were asked to complete five outcome measuresFootnote 5 at two time points—once prior to engaging in the assigned PLE, and a second time approximately 10 months later, following their participation in the PLE and teaching the instructional unit to their students. In addition, participants in all groups were asked to complete an instructional unit log after teaching their assigned unit. These six instruments are briefly described in Table 3.

Table 3 Quantitative measures
Full size table

Scoring rubrics were developed and pilot tested for each of the assessments. For the study, all indications of PLE format were removed from the responses before they were independently scored by two raters. There were four raters altogether; inter-rater reliability for all pairs was above 0.8. Discrepancies were resolved through negotiations between raters, but in the few situations when agreement could not be reached between the original pair, a final decision was made by a third rater.

Knowing that the experiences teachers in the various treatment conditions encountered in the three PLEs were quite different, a set of follow-up interviews was arranged to investigate how teachers attributed their learning to particular experiences in the PLE to which they were assigned. Twelve teachers (two from each research group) were chosen because their data exhibited notable gains from before to after participation in their assigned PLE. Each was invited to engage in a 2-h interview, which took place 2 years after participants completed the initial study. Nine of the selected teachers completed the interview. Before the interview, participants were mailed copies of their pre- and post-treatment assessments and asked to spend about an hour reviewing their work. During the interview, participants were asked (1) to reflect on what they learned from the PLE in which they engaged, (2) to comment on how their teaching had been influenced as a result of working within the particular PLE, and (3) to describe their instructional practices to illustrate the influences they identified. These participants completed two additional practice-based activities during the interview: one primarily addressing early algebra content (grades 1–3 teachers: 10 + 5 + 6 = 7 + 4 + 2 + 3 + m; grades 4–6 teachers: What is equal to 12x + 32 if 3x + 8 = 15? What about 12x + 30?) and a second addressing instructional practice for teaching early algebra (participants were asked to review a video of students working and subsequently to describe the students’ understanding of the mathematics). From the transcribed interviews and analysis of the participants’ work from both the pre- and post-PLE measures as well as the additional tasks, we wrote detailed profiles of the interviewees using a standard template. The profiles described the changes evident in the measures and the changes teachers described in their own knowledge, beliefs and instructional practice related to algebra content, use of context in teaching algebra, and observing students and conferring with them about their work. The profiles also detailed the specific opportunities teachers identified from their assigned PLE which they highlighted as influencing gains in their learning and changes in their practice.

2.4 Fidelity to PLE treatment conditions

Structural fidelity

All participants assigned to the facilitated workshop attended the entire summer workshop, verified by attendance records. Among those assigned to the multimedia course, 92% reported completing all of the assigned course work, verified by their submitted files. For those participants who were assigned to the curriculum resources, all reported reading parts of the materials and 76% reported that they had carefully read through the unit materials.

Teachers from all PLE treatment conditions were expected to teach their assigned curriculum unit during the 2010–2011 school year. Data from the unit logs indicate that the grades 1–3 teachers taught an average of 78% of their assigned unit, and that the grades 4–6 teachers taught an average of 77% of their unit.

Process fidelity

The three PLE conditions were very different in nature, and, despite the designers’ best intentions, it was not always feasible to verify that conditions were implemented as planned. The Facilitated Workshop followed all plans as intended. The two facilitators were very experienced and followed a detailed workshop plan (that was subsequently also published (Imm, Fosnot, Dolk, Jacob, & Stylianou, 2012)). However, the nature of the multimedia course, and the curriculum resources PLE conditions did not allow for direct observation of the processes participants followed when working in these formats.

2.5 Analysis models

Outcome scores for the questionnaire, content assessment, and practice-based assessments were analyzed using two-level hierarchical linear models (HLM), with the two time points nested within each teacher. The level 1 equation included one independent variable indicating which score was the post-test administration. The estimation of this level 1 variable was used to test whether post-treatment scores overall were significantly different than pre-treatment scores. The level 2 equations predicted the pre-treatment score and the post-treatment gain by PLE condition. Using PLE condition to predict the pre-treatment score controlled for any differences in initial scores across groups. To test for differences in gains among the three treatment conditions, treatment condition was used to predict the post-treatment slope, rotating the reference group in order to test all pairwise comparisons: facilitated workshop vs. multimedia course, facilitated workshop vs. curriculum resources, and multimedia course vs. curriculum resources.

Analyses were conducted for grade-level effects, as well as overall. That is, we performed tests between the three conditions at the lower grade level and then again between the three conditions at the upper grade level. We also performed an overall test that combined all participants together and looked for differences across time points for all quantitative measures.

3 Results

We began by mapping the three professional development formats to the six principles for effective professional learning as outlined earlier. Each of these formats for engaging teachers in professional learning included characteristics of what is known about effective professional development, as described in Table 4 below.

Table 4 How the three learning formats reflect what is known about effective professional learning experiences
Full size table

Notably, the facilitated workshop format encompasses more of the principles than the multimedia course and the curriculum resource PLE. Collective participation is absent from the latter two PLEs, as is expert facilitation in its traditional definition, but in a broader sense, both were designed for engagement by some of the same experts who led the facilitated workshop.

As a second step we examined the results from quantitative and qualitative measures. Here, we provide comparisons across several quantitative measures as well as sample responses to illustrate how participating teachers had grown from their professional development and teaching experiences in their understanding and teaching of algebra concepts. Table 5 below summarizes the quantitative results.

Table 5 Estimated pre- and post-treatment z-scores for participating teachers, by research group
Full size table

Impact on teachers’ beliefs about and preparedness for teaching the targeted algebra ideas

The teacher questionnaire asked participants to consider a set of ideas that might be addressed in early algebra instruction, indicating for each idea whether they considered it appropriate for students at the grade level they teach. These ratings were combined into an overall composite score of preparedness to teach early algebra. An overall change in the level of preparedness was found across the two time points (p < 0.05). Regardless of PLE assignment, teachers indicated that they felt more prepared to teach algebra ideas on the post-treatment questionnaire compared with the pre-treatment questionnaire in both grades 1–3 and 4–6 (large effects, respectively, of 0.97 and 0.81 standard deviations).

Impact on teachers’ content knowledge of the targeted algebra ideas

Teacher algebra content knowledge assessment scores were used to examine the impact of the three formats of professional learning on gains in teachers’ knowledge of targeted ideas. Differences in teacher content knowledge between pre- and post-treatment administrations were analyzed using the HLM model described earlier, with the addition of a variable in the level 2 equation indicating the teacher’s grade level (grades 1–3 or 4–6). Analyses included the 137 participants who completed both administrations of the algebra content knowledge assessment, revealing a statistically significant gain over time; post-treatment administration scores were higher than scores on the pre-treatment administration (p < 0.05), with a small effect size of 0.18 standard deviation units. Despite differences in initial scores, particularly between the two grade levels, no significant differences in gains were found over time between teachers in different experimental conditions or by teachers of different grade levels.

Responses to the open-ended content assessment questions illustrated various ways this growth in content knowledge was evident over time. Figures 1, 2, and 3 show pre- and post-treatment responses to the same question from grades 1–3 teachers from each of the three research groups. A commentary that provides interpretation of the responses can be found underneath each of the figures.

Fig. 1
figure 1

Facilitated workshop

Full size image
Fig. 2
figure 2

Multimedia course

Full size image
Fig. 3
figure 3

Curriculum resources

Full size image

The significant overall change in teacher content knowledge assessment quantitative scores was fairly small in statistical terms. However, across the three PLE research conditions, responses to the open-ended items, such as those shown in Figs. 1, 2, and 3, illustrated key shifts in teachers’ recognition of algebraic relationships and fluency in using them.

Impact on teachers’ knowledge of and ability to use instructional practices for teaching the targeted ideas

The three practice-based assessment instruments were used to address the study’s third research question. These three instruments simulated tasks of teaching that draw on knowledge of early algebra content and pedagogy. The use of context in teaching algebra assessment presented teachers with a narrative account that could be used to engage students with questions that required challenging mathematical work. Teachers’ scores for use of the context to support students’ mathematics learning improved over time for teachers at both grade levels, regardless of the PLE to which they were assigned. The effect size of this overall pre- to post-treatment gain was 0.50 for teachers in grades 1–3 and 0.49 for teachers in grades 4–6. For illustration, Fig. 4 displays pre- and post-treatment responses from an early-grade teacher who was assigned to use the curriculum resources. In the post-treatment response, this teacher shows a more sophisticated understanding of ways to make use of the context for engaging students with ideas in early algebra, as explained in the accompanying commentary.

Fig. 4
figure 4

Pre- and post-treatment responses from an early-grade teacher assigned to use the curriculum resources

Full size image

The analyzing student work assessment engaged teachers in watching video of a pair of students at work on a mathematics task form the curriculum unit and examining their written work. The assessment prompted teachers to analyze the students’ thinking and suggest ideas for conferring with these students to move their thinking forward. The videos presented to teachers at the two grade levels were different, so analyses were conducted within grade level as well as overall. For both teachers of grades 1–3 and 4–6, an overall pre- to post-treatment gain in scores was observed, with effect sizes, respectively, of 0.64 and 0.48 standard deviation units. Comparisons among teachers assigned to the three PLEs did not reveal differences.

Qualitative examination of individual teachers’ pre- and post-treatment responses provided evidence of differences in the mathematical details in what teachers noticed about the students’ work and in the specific questions they identified for conferring with the students to advance their thinking. Provided below (Fig. 5) is an illustrative example from an early-grade teacher, who completed the multimedia course, of the kinds of changes noted in teachers’ work on the video analysis task. Specific differences are described in the commentary.

Fig. 5
figure 5

Illustrative example from an early-grade teacher after completing the multimedia course

Full size image

The string of related problems assessment was designed to engage teachers in analyzing a sequence of problems that could be presented to students, with a focus on the content learning purpose of each problem and the sequence overall. The strings presented to teachers at the two grade levels were different, so analyses were conducted within grade level as well as overall. No overall gain nor differences by grade level or treatment condition were evident. Despite a lack of quantitative differences, in some cases important distinctions in the level of sophistication in responses could be identified when comparing pre- with post-treatment responses. For example, Fig. 6 shows an early-grade facilitated workshop participant’s deepened understanding of how the structure of tasks can be a catalyst for student learning, as described in the commentary.

Fig. 6
figure 6

Early-grade facilitated workshop participant’s deepened understanding of how the structure of tasks can be a catalyst for student learning

Full size image

Findings from the interview study

Taken together, the quantitative comparisons revealed a number of overall pre- to post-treatment improvements, but no statistically significant differences related to the PLE treatment conditions different groups of teachers experienced. The interviews and resulting profiles provide insight into participants’ perceived impacts of their assigned PLE on their knowledge, beliefs, and instructional practices in terms of algebra content, the use of context in teaching algebra, and observing students and conferring with them about their work. Summaries from three of the profiles, one from each of the grades 4–6 treatment groups, are described in this section. These three teachers were deliberately chosen to highlight here because they were in similar teaching situations at the time of the study and all three had shown substantial gains on several instruments.

Rick—facilitated workshop group

Rick (5th-grade teacher) was assigned to the facilitated workshop group. He credits the curriculum materials and discussions at the workshop as influencing all of his mathematics teaching: “Whenever I do math I think in a different way because of my learning through the materials and the discussion we had about math within that [workshop].”

Rick had large increases in his scores on the analyzing student work assessment task and self-reported preparedness to teach early algebra score. Rick stated in his interview that the opportunities the workshop provided for him to experience the unit materials as his students would and to engage in discussions with colleagues were vital to his improved feelings of preparedness.

The salient aspects Rick described about his experience in the workshop highlight several characteristics of what is known about effective professional development. He emphasized as especially helpful the opportunity to watch videos of students in order to diagnose students’ methods and conceptions, an experience that reflects an active, practice-based learning opportunity. Rick also pointed to the facilitator’s modeling of instructional techniques during his and the other teachers’ experiences with the algebra content at the workshop, noting an aspect of expert facilitation. Further, Rick identified the importance of coherence between the workshop and classroom teaching and content focus in the form of his experiences with activities from the curriculum unit. Finally, within Rick’s responses are references to discussions and conversations he recalled from the workshop, which suggest the importance of collective participation.

Karla—multimedia course group

Karla was a 6th-grade teacher who was assigned to the multimedia course research group. She completed the 25 h of coursework on her own. After completing the course and teaching the unit, she had a very large gain in her score on the use of context in teaching algebra assessment, and a moderately large gain in her string of related problems assessment score. When describing the parts of the multimedia course materials that she found most beneficial for her teaching, Karla singled out the videos and student work, and the opportunity to study them repeatedly:

For me it was watching the kids, honestly. Rewinding, watching the kids again, looking at their work, hearing their thought process, because that’s always key to me. Because I say to kids, you know, “Why? Why does that work?” and so to be able to rewind [the video] and see, okay, this is what they are thinking.

Karla also said that watching the students in the videos helped her to better understand the algebra content. Karla developed a better understanding of the double number line that was used to model equations and solve for unknowns. She also appreciated this representation as a useful tool for her students. In her words:

It was very helpful for me to hear kids ... use that double number line to visualize it…. Once you understood that double number line, which looks confusing, is really not once you understand it. It makes [the mathematics] very clear. It makes solving equations a visual—not that you would draw a number line every time you would solve them—but it gives you that visual picture that when you see that math problem, you know what it looks like.

Karla found her experience with the multimedia course materials particularly helpful in three aspects that reflect what is known about effective professional development. Primarily, Karla noted watching videos of students and examining student work, pinpointing the active, practice-based learning experiences of the multimedia course. Karla also expressed appreciation for the opportunity for multiple viewings of the videos and examinations of the accompanying student work, something that was supported by the recommended duration of the multimedia course, coupled with rich material that she found valuable enough to invest time in studying. She further tied these resources to her own learning about specific ideas in algebra and teaching algebra, emphasizing the content focus of the multimedia course.

Meagan—curriculum resources group

Meagan was assigned to the curriculum resources research group. At the time of the study, she taught 6th grade. She improved her score on all five of the research measures from the pre- to post-treatment administrations. When asked what aspects of the resources in the curriculum materials she found useful for her teaching, she pointed to the sample dialogs included for each lesson: “I think it was good to see ‘here’s what I’m looking for.’” She went on to say that the context presented in the unit was helpful for her teaching of algebra: “I think [the context used in the curriculum unit] gave me a real world idea of where can I go from here, what are the questions that I can ask, because the students understand it.”

One area that Meagan initially found challenging was using the suggested models for representing algebraic equations (e.g., the double open number line). She found the pictures and student work samples in the instructional materials very helpful for her own understanding, which was reflected in her increased score on the string of related problems assessment. Although the average score on this assessment essentially did not change across all teachers, Meagan’s score greatly increased. Like Rick, Meagan also exhibited a large increase in her self-reported preparedness for teaching early algebra score from pre- to post-treatment administrations of the teacher questionnaire.

Meagan’s reflections highlighted several features of educative curriculum materials that researchers have identified as supportive for teacher learning. Specifically, she linked her increased preparedness for teaching early algebra to the identification of questions that she might consider during instruction and descriptions of anticipated student responses that might arise in discussion. She also noted how the student work examples aided her preparedness for teaching by strengthening her understanding of representations of algebra content that are central to the curriculum unit.

4 Discussion

The purpose of this study was to investigate the impact of scaled-up, practice-based, and research-informed professional development addressing the knowledge and practices of teaching early algebra in the elementary grades, as experienced in three different professional learning formats. This is among the first random assignment large-scale studies of teacher learning from several types of professional learning experiences, albeit a narrowly constrained one around the specific topic of early algebra with related constrained learning goals and using particular designs for the three formats. Earlier studies had examined one PLE format, or compared and contrasted two PLE formats (e.g., Fishman et al., 2013; Krajcik & Delen, 2017) while this study extended that examination to three commonly used PLE formats.

Similar to earlier studies that conducted pair-wise examinations of PLE formats (Blanchard et al., 2016; Fisher, Schumaker, Culbertson, & Deshler, 2010; Fishman et al., 2013; Moon et al., 2014), this study finds no particular significant difference between conditions. Across multiple measures that we employed, we found evidence of learning in terms of teachers’ preparedness for teaching the targeted algebra ideas; teachers’ content knowledge; teachers’ knowledge and use of observing, questioning, and conferring strategies; and teachers’ knowledge and use of context and representations to support students’ learning. Comparisons of teachers’ pre- with post-treatment responses consistently indicated growth in learning for participating teachers, regardless of their assigned treatment condition, suggesting that all three formats of professional learning provided benefits to teachers. However, given the measures we used, no format stood out as more effective than the others. This finding is not surprising as it echoes and also extends earlier findings to encompass several PLE formats.

In follow-up interviews with a selected set of teachers who posted notable gains across instruments, we asked them to describe how their engagement in the professional learning format available to them supported their learning. In each case, these teachers highlighted aspects of the learning format that reflected the characteristics of effective professional development. To an extent, the study and the follow-up interviews provided evidence that each of the learning formats we tested was effective in supporting teachers’ learning, and the particular affordances and strengths of each format mirrored current understandings of effective professional development for mathematics teachers.

As we affirm earlier broad findings of no significant difference in gains between different formats, we turn our attention to other factors that might impact professional learning. As Kennedy (2016) noted, each PLE format is not monolithic or uniform. Rather, PLEs vary significantly across six elements as cited in the literature: (1) duration, (2) content focus, (3) coherence, (4) active/practice-based learning, (5) collective participation, and (6) expert facilitation. Our analyses suggested that all of these elements except collective participation characterized all three conditions as they were designed for this study. It is noteworthy that the workshops in this study were conducted in the summer with participants from various locations rather than being local, job-embedded workshops. Although “collective participation” was valued by the participants in the workshops, it did not appear to make a significant difference compared with the growth of participants who used the multimedia or curricular materials. This finding echoes earlier studies, particularly ones that examined online vs. face-to-face workshops. Fishman et al. (2013, p. 428) raised the question “Is there a loss in terms of building trust and local collegiality, or providing teachers with hands-on experiences?”. Collective participation, at least under the conditions of our design, might be the least scalable of the elements characterizing quality professional development, both in regard to logistical complications and cost. However, our study corroborates earlier findings, in that a lack of collective participation in alternative format did not necessarily diminish gains in professional learning and did not hinder the gains of online PLE efforts.

Few opportunities for teachers’ professional learning seem as scalable as what may be possible through well-designed educative curriculum materials. Most teachers draw on designated or supplemental curriculum materials almost daily in their instruction, so the potential for enhancing teachers’ learning during planning, instruction, and reflection through strategically placed resources is considerable. Beyond the affordances of written explanations of content and pedagogy, descriptions of connections among lessons and units, and samples of student work and classroom dialog, the digitization of curriculum materials offers ever expanding possibilities to provide these explanations, descriptions, and samples in more engaging and illustrative ways such as video, and more interactive ways through applets and social media. This study suggests that educative materials, when used as intended, offer opportunities for teacher growth potentially as helpful as other more costly and complicated professional development formats. We join authors of earlier studies in encouraging administrators, teacher educators and policy makers to view various PLE formats as potentially useful and effective in their contexts, and consider cost, feasibility, and content as factors at least as important as delivery format.

Limitations of the study and future directions

It may be argued that a key limitation of our study was the separation of the three PLE formats. Indeed, by separating the three formats, none of the teachers were exposed to what might be considered the most optimal learning experience (in-person workshop + guided access to the multimedia materials + use of the educative curriculum materials). Another option might have been to allow or even facilitate opportunities for participants in the multimedia course and curriculum resources condition to collaborate with colleagues (e.g., to create professional learning communities either in person or online for participants to collaborate). However, this separation was a deliberate design option. We wanted to see how each format could address the elements of professional development, and what impacts would result.

Hence, as a next step, we are interested in building and studying professional learning opportunities that blend together the supports each format affords could expand what teacher learning is possible. Educative elements of curriculum materials have been found to support teachers’ learning, in our study and others (Remillard, 2000), but considering the place of educative curriculum materials in hybrid formats for professional learning may further extend their potential. Dedicating some face-to-face or synchronous, online time to engaging teachers with examples of educative supports in their curriculum materials and how to utilize them could increase the probability that teachers will make use of them. Further scaffolding their use through guidance for individuals or groups of teachers working with them over time, similar to what a self-guided course provides, may further strengthen their effectiveness.

Another limitation of the study was that the design of the three conditions did not allow for detailed examination of fidelity of the process with which two of the conditions were implemented—the multimedia course and the curriculum resources. As it was designed, the research team relied primarily on participants’ self-reports, logs, and submitted files of assigned work. Future research might address fidelity to process more systematically by using online multimedia and curriculum resources that track user interactions, allowing for monitoring of time spent on, and engagement with, each activity.

The possibilities for expanded and extended professional development for mathematics teachers afforded by emerging learning formats are promising. At the same time, it is vital that we craft these new opportunities grounded in what is known about teachers’ professional learning and continue to conduct investigations that strengthen this knowledge base. Careful designs that build teacher learning opportunities from established and potential characteristics of effective professional development, coupled with research that informs and extends what we know about these characteristics, and identifies new characteristics that emerge, offer the strongest way forward to deliver on this promise.