Keywords

Researchers and teacher educators are increasingly interested in what teachers notice about their classrooms. Teachers who notice productively can target their teaching to students’ emerging mathematical ideas, as well as improve their instruction in ways that directly impact student learning (see, e.g., Fennema et al., 1996; Hiebert, Morris, Berk & Jansen, 2007). However, as Sherin, Jacobs, & Philipp (2011) note, “The study of noticing poses particularly thorny methodological challenges” (p. 11). One particular challenge is that researchers have operationalized this construct in a variety of different ways. In this chapter, we address one problem of operationalization—analyzing data. In particular, we suggest a method for analyzing teacher noticing data, which uses the mathematical learning goal of the lesson as a yardstick for analysis. We argue that mathematical learning goals are particularly useful as a generalizable method for conceptualizing and analyzing the mathematical nature of teachers’ noticing of student thinking across a variety of contexts.

In this chapter, we present a theoretical argument for why and how mathematical learning goals can be used to analyze teacher noticing data, using examples of such data to illustrate the process. Although researchers have used mathematical learning goals and their breakdown into conceptual parts (which we call “subgoals” or “key concepts”) to analyze data (e.g., Morris, Hiebert, & Spitzer, 2009), current research contains no methodological guidance on identifying subgoals and using them to analyze data. Most previous work on methodology for studying noticing has focused on the practicality of gathering data in the moment, looking at questions around the use of video cameras or the efficacy of recall (Sherin, Russ, & Colestock, 2011). Here, we do not focus on data collection; instead, we consider how to analyze the data once it has been gathered. We argue for an analysis framework for teacher noticing data that can further our understanding as a field and show how to better help preservice and practicing teachers develop expertise in noticing. In a metaphorical sense, we are arguing that most previous methodological discussions have focused on “attending” to teacher noticing—what researchers should look for in teachers’ work and how should we collect this data. In this chapter, instead, we are focusing on the interpretation stage. Once we have this data, how can we interpret it and respond to it in ways that move the field forward?

Definition of Noticing

There are multiple definitions and conceptualizations of noticing (Sherin, Jacobs, & Philipp, 2011). Each of these conceptualizations leads to different methodological choices for studying noticing (Sherin & Star, 2011), so it is important to clarify our definitions and focus. We define noticing as a teacher’s ability to identify, understand, and respond to student thinking in the midst of the distractions of the classroom (Sherin, Jacobs, & Philipp, 2011). While not all researchers take this stance, here we focus solely on noticing as a deliberate, individual process (Jacobs, Philipp, & Sherin, 2011). In particular, we define noticing as a discrete teachable skill that can be studied and improved through intervention (as suggested by emerging empirical work, e.g., Schack, Fisher, Thomas, Eisenhardt, Tassell, & Yoder, 2013).

We focus specifically on teachers’ professional noticing of children’s mathematical thinking in this chapter (Jacobs, Lamb, & Philipp, 2010). While other researchers have examined noticing more broadly (Jacobs, Philipp, & Sherin, 2011), the analysis framework we present is focused specifically on student mathematical thinking and we are not arguing it will work for broader conceptions of noticing. In addition, in contrast with some researchers, we focus on both what teachers notice and what they miss (Jacobs, Philipp, & Sherin, 2011). We believe researchers need to pay attention to what teachers fail to notice because what they fail to notice tells us as much, if not more, as what they notice. For example, a teacher who fails to notice important student mathematical thinking is unlikely to be able to respond appropriately in the moment or change their practices in beneficial ways (see Schoenfeld, 2011). The analysis framework based on mathematical subgoals that we describe in this chapter is particularly skilled at helping identify what was not noticed.

In this chapter, we focus on three key aspects of noticing: attending, interpreting, and deciding how to respond (Jacobs et al., 2010). The mathematics subgoal framework can help researchers analyze teachers’ ability to do all three skills. We take an expansive view of the skill of deciding, which various researchers have described in at least three different ways (see Table 1): responding in the moment with an instructional strategy (e.g., Jacobs, Lamb, Philipp, & Schappelle, 2011); seeking additional evidence to clarify students’ thinking (e.g., Schack et al., 2013); and reflecting back about how the lesson may have influenced students’ thinking and suggesting alternatives (e.g., Santagata, 2011). This third interpretation of deciding links noticing with research on teachers’ ability to learn from teaching (Sherin, Jacobs, & Philipp, 2011). The skill of learning from teaching is perhaps less required in the moment, where attending, interpreting, and deciding how to respond can influence immediate instructional decisions. However, we would argue that this skill is key to developing expertise in teaching and that these long-term decisions can be as important as short-term responses. Thus, professional noticing, where deciding is conceived as generating cause–effect hypotheses about the effectiveness of instruction, can help teachers generate new knowledge and improve their teaching over time (Santagata, 2011).

Table 1 Different interpretations of deciding
Full size table

Definition of Mathematical Learning Goals

We propose a methodology for analyzing noticing data focused on mathematical learning goals. Mathematical learning goals are statements of the mathematical content that students should learn in a lesson. They are similar to objectives in that they describe the outcomes of a lesson but different in that they do not need to be directly measurable and describe particular mathematical thinking rather than behavioral or observable student outcomes. Unlike standards, which are quite broad and long term, here we focus on short-term (one or so lessons) mathematical learning goals. These goals should be written in the language of mathematics (Hiebert et al., 2007). For example, a mathematical learning goal is better phrased as “Students will understand the balancing interpretation of the mean” instead of “Students will score 80% or more correct on an exam about the mean.” Specifying clear and precise mathematical learning goals has importance for many aspects of teaching, including selecting appropriate tasks (Smith & Stein, 2011), improving the effectiveness of teaching over time (Hiebert et al., 2007), and helping build a knowledge base for mathematics teaching (Jansen, Bartell, & Berk, 2009). We argue in this chapter that clearly specified learning goals can also be useful to researchers as a lens for analyzing teacher noticing data.

Once a learning goal is specified, we can make it more useful by unpacking that learning goal into its component parts, or “subgoals.” Subgoals represent the specific, but important, conceptual ideas that are necessary for a student to understand as part of the learning goal. Morris et al. (2009) argue that, “To be clear about learning goals means to identify the learnings required to achieve the goals” (p. 493). Subgoals, or key concepts of the learning goal, are different from prerequisite knowledge (which describe what a student must know before attempting to learn the goal) and instead attempt to specify as precisely as possible the mathematical ideas inherent in the goal. This skill is closely related to mathematics knowledge for teaching. This construct has been described by Ball, Thames, & Phelps (2008), who state that teachers “must hold unpacked mathematical knowledge because teaching involves making features of particular content visible to and learnable by students” (p. 400). Breaking a learning goal down into its key concepts is the process of stating those particular features and making them explicit.

Several previous studies have used learning goals and their component parts as a method of data analysis (e.g., Meikle, 2016; Morris et al., 2009; Phelps & Spitzer, 2012). For example, in a study of how prospective teachers might decide how to select and sequence students’ solution strategies during a class discussion, Meikle (2016) considered a learning goal about the division of fractions, and unpacked it into three component parts:

  • Learning Goal: Students will understand why the invert and multiply algorithm for division of fractions works according to the repeated subtraction meaning of division.

  • Key Concept A: Division can be interpreted as finding out how many groups of a certain size (the divisor) fit into the dividend.

  • Key Concept B: The reciprocal of the divisor represents the number of copies of the divisor that fit into one whole.

  • Key Concept C: The dividend represents the number of wholes we have. If the reciprocal of the divisor represents the number of copies of the divisor that fit into one whole, then we can multiply the reciprocal of the divisor by the dividend to find out how many copies of the divisor fit into the number of wholes (the dividend) we have.

In this example, the key concepts have the following attributes, which allow them to be useful to teachers and researchers. First, they break the learning goal down into constituent parts. These parts cannot be further divided and together make up the mathematics knowledge embedded in the goal. They also unpack important words and ideas in the learning goal (such as division and repeated subtraction) based on a deep understanding of the underlying mathematics. Although the key concepts are informed by research on developmental patterns of student learning, they are written in wholly mathematical language rather than describing pedagogical strategies or student behaviors.

Helping teachers themselves unpack a learning goal into its component parts has been shown to have the potential to help prospective or practicing teachers learn the skills of noticing (Morris et al., 2009), as well as other classroom skills such as selecting appropriate student responses to share during a whole-class discussion (Meikle, 2014). Collaborative work between teacher educators and teachers engaging in learning goal analysis together also has promise in improving instruction (Phelps, Shore, & Spitzer, 2014). Further research is needed to clarify and strengthen the links between teachers’ use of unpacking learning goals and their noticing skills. This chapter does not intend to provide such research, but instead to argue that this same process that teachers can use is helpful for researchers to analyze noticing data.

Why Use Mathematical Learning Goals to Analyze Data?

When teachers reference conceptual subgoals of a learning goal in their noticing, it indicates their attention to students’ reasoning about the important mathematical ideas of a lesson. Because the key concepts of a learning goal are unpacked into precise but important details, teachers’ attention to these concepts in students’ work reveals a mathematically sound and detailed analysis. This approach aligns with many others that have been used to analyze teacher noticing of student mathematical thinking in that it prioritizes detailed analyses which are mathematically relevant and in line with research about how children think and learn (Jacobs et al., 2010). This framework for data analysis has several advantages.

The primary advantage of using subgoals, compared to other analysis methods, is being able to more precisely specify teachers’ noticing. Many previous studies have analyzed noticing by focusing on the level of depth or detail in teachers’ responses. For example, Walkoe (2015) analyzes the depth of teachers’ conversations about students’ algebraic thinking using a level 0 code for conversations that discuss student thinking only generally and a level 1 code for those that include more depth and detail. Other researchers use a similar coding scheme focused on depth of analysis or level of detail of evidence (e.g., Choppin, 2011; Fernández, Llinares, & Valls, 2013; Jacobs et al., 2010). The use of mathematical subgoals builds on such an approach and allows a further level of precision about what (and how well) teachers have noticed. A subgoal focus can help researchers delve further into teachers’ noticing, sorting out the fine details that can distinguish between different “medium” or “high” depth responses.

This method also keeps researchers’ attention on the big mathematical ideas of a lesson, which is important for successfully responding in a way that improves student learning. Hiebert & Grouws (2007) argue that there is no such thing as “effective teaching” considered broadly, but only teaching which is effective at helping students achieve particular learning goals. Similarly, we argue teacher noticing is most effective when it relates to teachers’ mathematical goals for the lesson and helps students achieve those goals. For this reason, considering the level of detail alone may be problematic when studying teacher noticing because it is possible to be extremely detailed about unimportant student work or to provide little detail but be focused on important mathematical thinking. Instead, we can distinguish between higher and lower quality teacher noticing using its alignment to the learning goal, and specifically its alignment to the important conceptual ideas underlying that goal (i.e., the subgoals). This method of data analysis prioritizes the mathematical outcomes of a lesson over other pedagogical concerns, and could be used in concert with other coding schemes, which capture those concerns. Because other researchers have explored nonmathematical noticing (e.g., Erickson, 2011), we focus here only on using subgoals as a framework for analysis.

The additional precision granted by a subgoal analysis can help researchers attend to both what mathematical ideas teachers notice as well as how well they notice. This allows us to analyze both for what teachers notice and what they fail to notice, and helps us identify mathematical ideas that might be more difficult for teachers to notice. The coding schemes in prior research have primarily focused on what teachers notice; however, what teachers fail to notice is just as important because it is often these missed in-the-moment opportunities where growth in student learning could occur. Because a subgoal list attempts to identify all of the most important ideas about the mathematical learning goal, subgoals missing from an analysis often represent important mathematical ideas that teachers have failed to notice. Further, we have often found that some mathematical ideas are noticed by many teachers and others only by a few. For example, in our previous work, we have found that it is easier for PTs to attend to the ideas of “parts” and “wholes” than for them to notice when students are understanding the 10-times relationship between places (Phelps & Spitzer, 2012). Knowing what prospective and practicing teachers fail to notice or find hard to notice can help teacher educators better design professional development.

We also believe a subgoal analysis framework can help researchers recognize growth in teachers’ ability to notice. Jacobs, Lamb, and Philipp (2010) argue that several “shifts” in noticing of students’ thinking can demonstrate growth in expertise. They identify six particularly important shifts, including “a shift from general strategy descriptions to descriptions that include the mathematically important details” (p. 196). The use of subgoals is ideally suited for identifying such mathematically important details in teachers’ noticing that may signal growth shifts. Furthermore, subgoal analyses can be used across a variety of mathematical contexts and student work samples, allowing researchers to better compare teacher noticing across different tasks. By focusing on the mathematical content that we would like to see teachers notice, researchers can rely less on the particular, idiosyncratic features of a single lesson artifact.

Using Subgoals to Analyze Noticing Data

Having argued for the advantages of using subgoals as a method of data analysis, we will now describe how one might conduct such an analysis. We will use sample teacher noticing data to illuminate how subgoals are used and what affordances they allow. The sample data we present here was collected as part of a study of how an online discussion board assignment might help preservice teachers (PSTs) learn to notice, analyze, and learn from student thinking in a sample lesson. We do not share results of this study here (see Spitzer & Phelps, 2011 for further details), but instead to use this data as an illustrative example of the subgoal process. Participants consisted of 16 PSTs enrolled in an elementary mathematics methods course. As part of a course assignment, participants watched the video “Meter Cords” from the Annenberg Collection (WGBH Boston, 1997). In the video, third and fourth grade students are shown being successful at a class activity (hands-on measurement), but reveal serious misconceptions about the learning goal (understanding decimals). The learning goal for the lesson (as stated in the instructions to PSTs) was: “Students will understand that decimals represent parts of wholes; in particular, the tenths place represents quantities which are ten times smaller than the quantity represented by the ones place.”

After watching the video, PSTs interacted in an online discussion board that was formatted as a “debate” about the lesson’s effectiveness in helping students achieve the learning goal. After the “debate,” PSTs wrote a reflective essay (see Spitzer & Phelps, 2011, for results about the effectiveness of the online discussion board intervention). Later in the course, PSTs would be explicitly taught how to unpack a learning goal into its component parts (as well as other skills of teacher noticing), but this data represents their initial, untrained ability. This data provides a rich site to study teacher noticing because the debate and essay prompts related to all three skills of noticing (attending, interpreting, and deciding) and involved a video which provided “windows into student thinking” (Sherin, Linsenmeier, & van Es, 2009, p. 215) as well as a variety of distractors. We will describe how we used the mathematical learning goal to analyze this data through three main stages: creating the subgoal codes, using them to code data, and making claims from the codes.

Defining the Key Concepts and Subgoal Codes

To analyze data based on the learning goal, the first (and often most challenging) step is to define the subgoals of the learning goal. In order to identify and unpack these subgoals, it is useful to begin with a theoretical consideration of what mathematical ideas are inherent in the goal. Empirical research into students’ thinking about the particular goal can also be helpful in identifying subgoals. For example, if considering a learning goal related to solving addition and subtraction word problems, the work of Cognitively Guided Instruction (e.g., Fennema et al., 1996) would be a useful resource. Learning trajectories (see, e.g., Clements & Sarama, 2004) can also provide insight into the component parts of a learning goal. Initial subgoal lists can be refined through iterative reading of sample teacher noticing data or through pilot studies and should ideally be checked for validity with an outside expert. Like any qualitative data analysis, valid methods require a back-and-forth iterative process between the data and the codes based on the subgoals of the learning goal (Miles & Huberman, 1994).

For the learning goal associated with our lesson sample, we identified the following component parts (see Figure 1). We feel that this list captures at least a minimum of the important mathematical ideas of the learning goal because we developed it based on the work of previous researchers (see, for example, Morris et al., 2009). In addition, we kept refining the key concepts until we were in agreement, in much the same way that qualitative researchers refine qualitative codes until there is high interrater reliability.

Figure 1
figure 1

The learning goal and identified key concepts.

Full size image

In addition to the specified subgoals, it is usually necessary to record additional features of student thinking noticed by teachers. These primarily fall into four categories: claims related to the learning goal, but not detailed enough to be considered a key concept; claims related only to procedural competence, even though the learning goal is conceptual; claims irrelevant to the learning goal; and claims that are primarily pedagogical (not mathematical) in nature. When using a subgoal analysis, we also use codes for each of these four categories (relevant but vague, procedural only, irrelevant, and nonmathematical). Making a record of what teachers did notice (aside from the important mathematical concepts of student thinking) allows us as researchers to capture shifts in noticing more completely and will aid in quantitative comparisons.

Using the Key Concepts to Code Noticing Data

Our conception of teacher noticing of student mathematical thinking follows Jacobs, Lamb, and Philipp (2010), who describe three component skills: attending to student strategies, interpreting those strategies in terms of student thinking, and deciding how to respond on the basis of the noticed thinking. The identified subgoals of a learning goal can be used to code and analyze data from these three skills. We will now describe and justify the use of a subgoal analysis for each of the three skills, using examples from the study described above to illuminate how subgoals can be a useful lens for teacher noticing data.

Attending. According to Jacobs, Lamb and Philipp (2010), the cornerstone of teacher noticing is attending, which focuses on “the extent to which teachers attend to a particular aspect of instructional situations: the mathematical details in children’s strategies” (p. 172). As we have argued, the use of mathematical subgoals provides a valid and generalizable way to identify what the important mathematical details are for a given learning goal. The use of mathematical subgoals allows researchers to distinguish between teachers who attend with a high level of detail to irrelevant mathematics from those who attend with a low level of detail to important mathematical ideas.

Consider the following two examples, both written by PSTs in response to the prompt: “Was the lesson successful in helping students achieve the learning goal? Cite evidence of what students do or say that leads you to believe that the lesson either helped or did not help students achieve the learning goal.” Beth wrote:

The children were first asked to create measuring sticks out of string. They then laid the string parallel to a meter stick and placed a red piece of tape at every tenth of a meter. They were then asked to take the string and measure various things throughout the classroom. After collect[ing] all of their data, the students were asked to create charts or graphs that represented the data they found. All groups of students were able to correctly measure the objects and create a chart that accurately represented that data.

In this response, Beth provides a high level of detail about students’ actions, but does not address students’ understanding of the learning goal (see Figure 1). We coded this as “Irrelevant math (measurement and data analysis).” Contrast this response with Felice, who wrote:

When the group of students was measuring the width of the desk they thought it was a meter and a twelfth not a meter and two tenths. They looked on the chart for where a meter and a twelfth was but it wasn’t there. They saw 1.2 so I believe they thought it was the right answer because without the decimal point it would be the number twelve.

Felice focuses her attending on a conceptual idea inherent in the learning goal, specifically the differences between the numbers 12 and 1.2 (coded as a reference to Key Concept 1; see Figure 1). The presence of this subgoal reference indicates a higher quality analysis than Beth’s, even if the level of detail is the same (or less). Attention to students’ strategies around multiple subgoals would indicate an even higher quality response.

Interpreting. Once teachers have attended to the details of student strategies, they must decide what those details tell them about students’ learning. This skill of noticing is particularly well suited for a subgoal analysis, since its goal is to determine the particular mathematical ideas (e.g., key concepts of the learning goal) that students understand (or do not understand). Additionally, interpreting has been particularly difficult for researchers to operationalize in previous work. For example, Schack et al. (2013) highlight the difficulty of quantifying the depth of analysis for the interpreting case when they write, “Interpreting presented the most difficult set of benchmarks to construct due to [prospective teachers] focusing on different aspects of the child’s work and/or different, yet reasonable aspects of the mathematics” (p. 389). The use of subgoals can help both clarify and quantify the aspects of the mathematics that teachers notice.

For example, consider the responses of Haley, Beth, and Alyssa (see Table 2). All three of these examples might be considered “medium depth,” but while Beth and Haley notice details related to student thinking about the learning goal (in different dimensions, but of comparable quality), Alyssa only considers surface features of the lesson and students’ procedural competence rather than conceptual understanding. The use of details in Alyssa’s quote do not indicate useful insight into student achievement of the learning goal, reflected in the fact that she does not reference any subgoals. The use of subgoals to distinguish between these three examples helps us as researchers tease out what mathematical ideas PSTs are able to interpret in student thinking as well as distinguish between better and worse responses. Thus, a subgoal analysis can help researchers distinguish between different “medium” or “high” depth responses, recognizing depth does not imply relevance and brevity does not imply lack of important mathematical ideas.

Table 2 Interpreting responses of three PSTs
Full size table

Blending the skills of attending and interpreting. One difficulty that we have observed when studying teacher noticing is that, when discussing student thinking, it is natural for teachers to blend together their attending and interpreting. Rather than organizing their thinking as first describing everything that students did, then interpreting it, teachers may instead alternate between describing pieces of students’ work and immediately making inferences about student thinking. This is a challenge across the field of teacher noticing, as pointed out by Sherin and Star (2011), who note that “Researchers may be unable—both in practice and in theory—to separate the earlier steps of the intuitive model, that is, to separate noticing from interpreting” (p. 70). Consider the work of Danielle, who organized her response around Key Concept 4 (see Figure 1) and first attends to and then interprets a student response:

When the students measured the door, they said it was twenty tenths longs. The students did not realize that this measurement really means, 2 meters. The students were unable to grasp the concept of how the method of the “tenths” worked. If they did understand the learning goal of the lesson, they would have known once they got to a full meter, also known as “10 tenths,” they would have to start counting over. For example the door was twenty tenths long, this means that the students should have counted 10 tenths on the meter rope, then started over and counted another 10 tenths on the meter rope which would then give them 2 meters in total.

This example illustrates how a teacher can mix attending (lines 1-2) and interpreting (lines 3-4), together with what they notice about what a student does not say (lines 4-9), into an argument about what the child knows about a particular key concept of the learning goal (coded as a reference to Key Concept 4: that in a number written in standard form, each place can only “hold” a digit from 0-9). That key concept makes a natural unit of analysis.

Furthermore, just as a subgoal analysis can help us as researchers analyze what teachers do not notice, it can also help analyze teachers’ responses when they discuss what children do not say, which is an important feature of noticing (Ball, 2011). The hypothetical case that Danielle presents (what a student would have done differently if they had understood this key concept) is a useful act of noticing even if it is not about any specific student work in the classroom. By using the subgoals of a learning goal to analyze responses such as these, researchers can capture the blended skills of noticing.

Deciding. Deciding is the act of using what is noticed about student thinking to make an instructional response. According to Jacobs, Lamb and Philipp (2010), teachers’ deciding should be evaluated based on “the extent to which teachers use what they have learned about the children’s understandings from the specific situation and whether their reasoning is consistent with the research on children’s mathematical development” (p. 173). The use of subgoals can help researchers analyze teachers’ decisions along these two dimensions. For example, Beth wrote:

If I were teaching this lesson, I would not correct the students if they were wrong, rather I would have them explain to me how they came to their conclusion. I would hope by doing this, students could self-correct their mistakes. I do not think it benefits a student to be told that they are wrong. I think it is more beneficial to the students to correct themselves, if possible. This way they will hopefully learn from that mistake.

A focus on subgoals allows us to see that, although Beth’s response is aligned with research recommendations for teaching and appropriate pedagogically, it does not address any important ideas of the learning goal (no key concepts are present), and thus does not use the specific situation of the lesson.

Previous research has struggled with analyzing the skill of deciding because often prospective teachers’ proposed revisions to lessons are aligned with what they have noticed about student thinking, but only target procedural competence or tangential aspects of the lesson (e.g., Jansen & Spitzer, 2009; Parks, 2008). The use of unpacked learning goals allows researchers to record whether teachers’ decisions would address the mathematical ideas of the learning goal. For example, Kaylee addresses her decisions to irrelevant mathematics (measurement and data analysis), writing:

The only thing I would add to the lesson was more specification on the objects that were being measured. Many of the students were unsure where to start and stop measuring which resulted in different data on the bar graphs. By being more specific the teacher may have been able to avoid this situation and get a better representation of data for the class to look at.

Contrast that response with Marisa, who proposes a response to student thinking based on Key Concept 4 (see Figure 1):

I think that [it would have been helpful] if the teacher had demonstrated measuring an object that was longer than a meter and counted out loud, “7, 8, 9, 10, ok now once we get to ten that equals 1 whole and then we start counting again to the next whole”. Then if she also showed that on a number line on the board I think it would have helped as well. Then she could have shown where the 1, 2, 3, etc. were and then the tenths in between the whole numbers as well.

The use of subgoals can also help researchers determine which ideas teachers tend to target for their revisions. This is one way teacher noticing data could be used to help build a knowledge base for education (see Santagata, 2011).

As mentioned above, we consider that deciding can take several forms, including proposing a next step, seeking more information on student thinking, and using cause–effect hypotheses (which reflect back on the lesson to link teaching moves with student learning outcomes) to suggest lesson revisions. All of these are appropriate ways to respond to student thinking (either in the moment or after the fact). In particular, constructing hypotheses and suggesting lesson revisions allows teachers to use what they have noticed about student thinking to learn from and improve their teaching over time (see, e.g. Hiebert et al., 2007; Santagata, 2011). For example, Felice describes a lesson revision centered around Key Concept 2:

The teacher should have modified the lesson by having two different color tapes to represents “parts/decimals” and “wholes.” While the students were counting tenths they would use the red tape and once they got to a whole/ten tenths, they would mark it with blue tape. I believe this would help them better understand that the tenths place quantities are ten times smaller than the quantity represented by the ones place.

Or consider Corinne’s deciding response, shown below, which contains both a thinking-back hypothesis about what features of the lesson might have led to the student thinking she noticed as well as a targeted revision to those lesson features. Her references to Key Concept 4 demonstrate the alignment of these hypotheses and revisions to the learning goal and observed student thinking:

I also believe that the comparison the instructor made about quarters and tenths as parts of a whole was confusing. She tried to relate money to measurements so the students would understand to use decimals, but this seemed somewhat confusing and the students did not relate this to the topic at hand. It could have been helpful if it was explained more clearly.

As Schoenfeld (2011) states, “What makes noticing consequential, of course, is that people act on what they notice” (p. 230). When teachers’ decisions about how to respond to student thinking are aligned with and targeted to key concepts of a learning goal, these decisions are more likely to lead to improved student learning of those goals. Thus, it is important for us as researchers to use a coding scheme that can capture alignment with key concepts of the learning goal.

Using the Codes

Once the data has been analyzed, the task remains of using the codes to make conclusions about teacher noticing (for example, whether teachers have improved their noticing skills from an intervention, or how noticing might compare across different populations of teachers). For small-scale qualitative studies, this might simply entail a close inspection of the subgoals noticed by individual participants. For much research, however, it is useful to envision ways in which subgoal data might be used quantitatively to make judgments about noticing. In order to illustrate some potential avenues for analysis, we will describe some of the methods we have used in our own work; of course, data analysis is a creative process, which must take into account the complexities of each individual context.

Making comparisons across individuals. One primary way in which researchers might want to use subgoals is to make comparisons across individuals, possibly to compare between different teachers (or populations of teachers), or to show whether an intervention was effective at improving noticing skills. In order to do so, it is necessary to be able to use the subgoals to identify areas of strength and weakness in individual teachers.

Throughout this chapter, we have argued that higher quality noticing work references more subgoals of the mathematical learning goal. Thus, the first step of analysis is to look at teachers’ coverage of the subgoals, that is, how many of the identified subgoals did they reference in their noticing? Obviously, in this case, noticing responses that reference more subgoals are better responses, so the number of subgoals addressed can represent the quality of noticing. For example, in the study we describe in this chapter, PSTs referenced a mean of 1.73 (SD = 1.4) key concepts (out of four) on the pretest and 2.33 (SD = 1.1) on the posttest (p < 0.05). This indicates that the intervention was successful in helping PSTs notice a full range of student thinking around the learning goal.

Another important feature of teachers’ noticing is the level of alignment of their response across the three skills (attending, interpreting, deciding) in relation to the subgoals. Higher quality responses include attention to all three skills around the same subgoal (as opposed, for example, to making a response decision that is aligned to a different subgoal than was attended and interpreted). In the study described here, on the posttest, 80% of teaching decisions related to a key concept of the learning goal were aligned with an attending or interpreting response around the same key concept, compared to 64% on the pretest (however, due to a low number of such decisions, this difference was not significant).

Recall that in addition to the subgoals, we also code for four other types of noticed student thinking: mathematically relevant but unspecific ideas (e.g. “She understands decimal concepts”); irrelevant mathematical ideas; procedural skills; and nonmathematical behaviors (e.g., “she was highly engaged in the task”). We use these non-subgoal codes to indicate differences between individuals or change over time. These non-subgoal codes provide an overall classification of what a teacher noticed, allowing us to look for differences across categories. For example, a higher percentage of claims that were aligned with a subgoal (as opposed to a higher percentage which were irrelevant) would indicate a higher quality analysis. In the study described in this chapter, we found that among a subgroup of PTs, the percent of claims that referenced a subgoal rose from 29% initially to 52% after the intervention (p < 0.01), providing evidence that the intervention was successful for that subgroup (see Spitzer & Phelps, 2011).

Making claims about populations. In addition to these claims about individuals, researchers can also use the subgoal codes to make arguments about populations of teachers. As noted above, the use of mathematical subgoals can help researchers and teacher educators tease out what mathematical ideas are easier or more difficult for teachers to notice, and design interventions accordingly. For example, after the intervention in the study described above, we found that many more PTs (12 out of 16) made claims mentioning Key Concept 3 (wholes vs. parts of wholes on different sides of the decimal point) compared to Key Concept 1 (digits in different places have different value; noticed by only 1 PT). This aligns with previous research (e.g., Morris et al., 2009) which suggests that PTs often attend only to the most prominent features of student thinking, such as a wrong answer to a problem, while failing to notice more subtle clues about the sources of students’ misunderstanding. This and similar findings can help teacher educators design interventions to help teachers learn to notice more difficult mathematical ideas as well as look more deeply to uncover hidden misconceptions.

Conclusion

In this chapter, we have argued that when what teachers notice is aligned with specific, important conceptual details of the mathematical learning goal, they will be more likely to respond appropriately in the moment and make productive changes to their teaching over time. Thus, it is possible for researchers to use this alignment to key concepts of a learning goal as a generalizable way to analyze teacher noticing data across different contexts and subjects. The work of analyzing teacher noticing and using this construct to improve teaching is a young but emerging field (Sherin, Jacobs, & Philipp, 2011). We hope that our methodological description will help guide this field in productive ways.