1 Introduction

The ongoing tension between extant instructional practice and potential instructional improvement (or reform) underscores the importance of investigating the decisions that teachers of mathematics make in practice. What is in play for a teacher when usual practice conflicts with potential reforms? On the one hand, the teaching of a course of mathematical studies to students of a given age group is a specialized practice to which one might expect teachers become progressively socialized, with each new class of students providing a new opportunity to engage in recurrent types of mathematical work. This socialization can be represented in terms of norms—shared expectations of teachers of how instruction routinely proceeds, which an observer might also document as recurrent patterns of classroom interaction. On the other hand, the positions in which teachers of mathematics work require them to be responsive to pressures from a range of sources to which mathematics teachers, as professionals, are obligated. Quite often, those sources pressure teachers to teach differently. These pressures include, for example, the responsibility to make students’ mathematical experiences better aligned to disciplinary practices, the responsibility to create more differentiated opportunities for individual students, or the responsibility to increase the effectiveness of educational programs.

As teachers are called to act in new encounters with students, it is reasonable to expect their decisions to be influenced by different types of resources. Some of these resources are individual and include teachers’ own knowledge of mathematics, their epistemological beliefs, or their personal goals; such personal resources are often considered in accounts of teachers’ decisions (Schoenfeld, 2011).

Other resources are provided by the institutional contexts where teaching happens (Chazan, Herbst, & Clark, in press). Herbst & Chazan (2012) called attention to instructional norms and professional obligations as contextual resources that matter in teacher decision making. The notion of instructional norm brings attention to a teacher’s socialization into the activity systems in which they customarily interact with students and content. Herbst & Chazan (2012) describe contract Footnote 1 and situation Footnote 2 as two activity systems whose norms inform the actions a teacher takes. Thus norms are one set of sociotechnical resources that we hypothesize a teacher makes use of when making decisions as to what to do. The notion of professional obligation brings attention to a teacher’s position as a professional. Herbst and Chazan (2012) identify four sources of professional obligation for the mathematics teacher (the discipline of mathematics, the individual child, social life, and the institutions of schooling). The professional obligations span sets of values that inform what a teacher chooses to do, particularly lending attention to issues to which the teacher may not be personally inclined to attend.

That to model teacher decision making requires researchers attend to both the individual as well as the contextual resources teachers use to make decisions is apparent in the case of curriculum implementation. The curricula developed after the NCTM Standards (NCTM, 1989, 2000) have sought instructional improvement by infusing novel mathematical work for students to do, work that aims to engage students in practices that resemble the work of mathematicians (e.g., problem solving, reasoning). These curricula seek to disrupt the routine contexts of mathematical work that we call instructional situations; yet there is variation in the extent to which curricula are enacted (Kisa & Correnti, 2014). In the literature on curricular decision making, that variation has often been explained in terms of individual factors (Remillard, 2005). The notion that individual teachers are the agents of reform-guided instructional improvements is a theme in mathematics education scholarship (Battista, 1994). Remillard and Heck (2014) identify factors that influence the enacted curriculum: “teacher and student knowledge, beliefs, and practices, access to resources, such as instructional technologies, and contextual opportunities and constraints” (p. 714; see also the CSMC curriculum research frameworkFootnote 3). Though from our perspective that list of factors is too focused on the individual teacher, we note as an important point of reference the mention of “contextual opportunities and constraints” (p. 714).

In this paper, we provide means to explore how individual and contextual resources in the institutions where teachers work inform decision making. We do that by showing instruments we have designed to explore norms and obligations, as well as how we have designed instruments that elicit teachers’ decision-making at moments when an instructional norm might be at play. Those instruments are designed in the service of answering the following questions: When enacting an instructional situation, that is, in the context of managing subject-specific work in a course of studies, and upon the opportunity to act according to a norm, do practitioners choose to act according to the norm or to deviate from it? How can we account for the decisions that teachers make when they do and do not choose to act according to the norm?

In making an argument for the importance of designing instruments that can empirically investigate teacher decision making at scale, we draw on contemporary practice theories (e.g., Bourdieu, 1990; Nicolini, 2012) that speak against the reduction of human action to either individual agency or social structure. We hypothesize that the decisions that teachers make are products of how individuals use personal resources to negotiate the demands of their institutional positions and the norms of the activities in which they play roles. Instead of inquiring into the cognitive processes a subject uses to act and decide, we inquire into the individual and contextual resources that play a role in the decisions that are made. Those resources include explicit knowledge that might assist an actor who consciously holds it in deciding what to do. But the resources used to make decisions also include other elements of knowledge that are inscribed in the institutional structure in which the actor takes a position or in the instructional situation the actor needs to manage: Such resources might be described as collective tacit knowledge (Collins, 2010) and they could include things such as a situated disposition to attend to particular matters.

2 Literature review

The decisions teachers make in classrooms can affect students’ opportunities to learn. That teacher decisions can augment or diminish such opportunities highlights the importance of understanding the basis on which these decisions are made. Schoenfeld (2011) reported a cognitive model of teachers’ decision making, where teachers as rational actors make decisions based on their own goals, beliefs, knowledge, and orientations (Schoenfeld, 2011). An earlier approach spearheaded by Bishop (1976; Bishop & Whitfield, 1972) suggested that teachers’ decisions might also depend on the situations in which a teacher encounters him or herself.

The broad literature on human behavior has been concerned with explaining cognitive processes that go into decision making. Prospect theory was developed by Kahneman and Tversky (1979) to account for common biases that were uncovered during their empirical study of decision-making. This theory is largely restricted to situations that can be described as wagers based on set probabilistic outcomes. Social judgment theory (Hammond, Stewart, Brehmer, & Steinmann, 1986) is based on the idea that people have to make their decisions based on environmental cues that allow them to make predictions about what is going to happen in the future. To better understand how people make decisions, these researchers create regression equations that account for the relative weight those cues are given in the decision-making process. But this theory does not actually provide any rationale for why particular cues matter, instead employing data collected from an individual’s history.

In psychology, the notions of System 1 and System 2 proposed by Kahneman (2002) maintain the decision as an individual phenomenon, but attribute the rationality in making such choices to qualitatively different mechanisms. We acknowledge that to the extent that an individual is the material carrier of actions, individual mechanisms have to be involved. Yet recent advances in behavioral economics (e.g., Ariely, 2010) suggest that the decisions individuals make respond to more than the personal resources of individuals: Conformity to social norms sometimes trumps the individual’s rational response. March (1994), writing from a management perspective, encapsulates the difference between the personal and the social justifications for decisions by distinguishing a logic of consequence from a logic of appropriateness:

A logic of consequence encourages thought, discussion and personal judgment about preferences and expectations; a logic of appropriateness encourages thought, discussion, and personal judgment about situations, identities, and rules. Both processes organize an interaction between personal commitment and social justification (March, 1994, p. 101).

Thus, regardless of the nature of a particular decision, individual and contextual resources inform the decision. Doyle and Ponder (1977) describe a similar interaction in direct reference to teachers’ decision making by suggesting that it is the interaction between the ability to apply, or translate, certain decisions into practice, as well as relating it to particular context of teaching that facilitate the actualization of decision making among teachers. Present in March’s (1994) description, but less explicit in Doyle and Ponder’s (1977) is the incorporation of social justifications in the decision making process.

Shavelson and Stern (1981) addressed the difficulty of accounting for the rapid decision-making required of teachers through the use of schemas, or models, that are used by teachers to account for their students, the task, and the context when a pressing issue required them to make a decision. Further, they believed “that teachers idiosyncratically build up these schema over time and practice” (Borko, Roberts, & Shavelson, 2008, p. 47). Various other contributions to the literature on teachers’ decision making have described the importance of context. Brown and Coles (2011) draw on enactivism, the idea that teacher decision-making is not individual but rather that “the culture of a classroom can be viewed as emerging over time from the patterns of social interactions between teacher and students” (p. 862). Skott’s (2004) description of how curriculum standards can operate as a set of demands or obligations on teachers—rather than allowing for their pedagogical autonomy—suggests that curricular demands have the potential to interact with teachers’ decisions given particular contexts. Similarly, Stoffels (2005) used a case study to argue that the decision-making of science teachers in South Africa could “at best be described as limited and passive” (Stoffels, 2005, p. 539) due to increased institutional demands.

As argued by the foregoing brief display, the literature on teacher decision making has been dominated by accounts of the influence that individual cognitive factors, such as knowledge or beliefs, have on how teachers make decisions. But, from Bishop and Whitfield (1972) to the present, there has not been substantial progress in examining how the context in which decisions are made provides information to understand what decisions are made. We tackle the question of how to account for the decision making resources provided by the context, what they are, and how they matter.

3 Constructs that identify contextual resources for decision making

We seek to account for how an individual teacher manages making decisions in the stream of instructional action. We expect that there will be two sets of resources that the individual will draw upon: one set that comes from within the individual and another set that comes from the context the individual is in. While our goal in this paper is to describe instruments that we have designed for such an account, the descriptions of the instruments are preceded by conceptualizations of the constructs they measure. We do not dwell at length on individual resources, but rather refer the reader to Schoenfeld (2011) for a theoretical account of these. We also count on the availability of instruments to measure teachers’ individual resources such as mathematical knowledge for teaching (e.g., Ball, Thames, & Phelps, 2008; Herbst & Kosko, 2014; Hill, Ball, & Schilling, 2008) or beliefs about mathematics and its teaching and learning (Stipek, Givvin, Salmon, & MacGyvers, 2001).

3.1 Contextual resources

Our proposal for understanding teachers’ decision making brings to existing research on teacher decision making an account of resources provided by the context in which the individual teacher is acting and relies on our evolving account of the practical rationality of mathematics teaching (Herbst & Chazan, 2012). Practical rationality alludes in general to the habitus, or system of categories of perception and appreciation, that characterizes the particular feel for the game that teachers of mathematics have (Bourdieu, 1990). The expression only points to a coarsely defined phenomenon that is hypothesized to exist: We contend with it that practitioners of mathematics teaching act in ways that appear regulated by a common rationality, a common way of perceiving and appreciating what is sensibleFootnote 4 to do. The naming of that space enables theory-building research which includes proposing more precise constructs, constructing instruments that measure them, and using those instruments to ground the hypothesis.

3.1.1 Instruction and its norms

Our account of the practical rationality of mathematics teaching has proceeded in three stages. In the first stage we developed means to describe mathematics instruction as a set of discrete practices of managing symbolic exchanges. In these instructional exchanges, the teacher organizes mathematical work for students to engage with mathematical ideas at stake, and the teacher also takes the enactment of mathematical work as evidence of students’ knowledge of those ideas. One can think of a course of studies (e.g., the Euclidean geometry studied in US high schools) as articulated by a list of objects of knowledge at stake and the work of the teacher as one of enabling students to do mathematical work based on which they can lay claim on those objects. In addition to organizing work for students to do, the teacher also allocates value to their completed work, by figuratively exchanging that work for a claim on the students’ relationship to the knowledge at stake.

In the first iteration of this work, we created means to model those exchanges between mathematical work and knowledge claims: To model them means to represent them as the enactment of norms from subject-specific instructional situations. For example, the instructional situation “doing proofs” includes the norm that every statement in a proof must be justified by a reason (Herbst, Chen, Weiss, & González, 2009). The notion that some actions in the execution of mathematical work are informed by (in the sense of being in the domain of application of) some norms is key in our approach to modeling instruction and in what we can say about decision making at this time. These norms are mathematically specific but not purely mathematical—they concern instructional actions, namely interactions among teacher, student, and mathematics (Cohen, et al., 2003). In the example provided above, the norm is not only that a reason must exist for a statement to be made, but actually that a reason must be written out. We hypothesize that courses of study have many instructional situations (Chazan, 2013) and these rely on manifolds of norms that suggest preferred courses of action specific to each instructional situation; these courses of action are modeled by stipulating norms.

In spite of the fact that instructional norms establish some preferred courses of action, mathematics teachers do not always act accordingly (norms are neither obligatory like civil laws, nor ineluctable like physical laws). We have been interested in accounting for mathematics teachers’ decisions in such context. As noted in the prior paragraph, a key sociotechnical resource is the instructional situation with its set of norms.

3.1.2 Professional obligations

To search for sources of reasonableness, we observe that teachers do not merely play roles in managing exchanges between work and knowledge but, more broadly, they occupy a professional position that is accountable to four educational stakeholders: to the discipline of mathematics, to children as individuals, to community and social life, and to the institutions of schooling. We have argued that mathematics teachers are professionally obligated to these stakeholders and that these professional obligations provide a basis for justifying the decisions teachers make. Obviously the motivations for why somebody might have done something can be idiosyncratic, even selfish, but we contend that for those actions to be seen as appropriately professional, they also need to be seen as justifiable or reasonable. The four obligations name in general four perspectives to which a professional mathematics teacher can be held accountable (Chazan et al., in press).

3.2 Decision making in instructional situations

The decision making problem we have been interested in can now be formulated more narrowly as follows: When enacting an instructional situation and upon the opportunity to act according to a norm, do practitioners choose to act according to the norm or to deviate from it? How can individual and contextual resources help account for the decisions that teachers make when they do and do not choose to act according to a norm? In what follows we describe efforts to assess, on the one hand, decisions teachers make in instructional situations and, on the other, hand their recognition of norms and obligations.

4 Instruments

Our group has been working with three instructional situations, one in geometry (doing proofs; Herbst, et al., 2009) and two in algebra (solving one variable equations and solving word problems; Chazan & Lueke, 2009; Chazan, Sela, & Herbst, 2012). Our decision instruments concern one norm in each of the instructional situations in algebra and two norms in the instructional situation of doing proofs in geometry. Below, we briefly describe the instruments we have developed for norm recognition, and recognition of the four professional obligations. And we dwell at length on the design of our decision instruments, including some results from pilot data. In particular, we describe decision and norm instruments in the context of one of the norms in geometry: The norm that it is the teacher who provides the ‘givens’ and the ‘prove’ for proof problems. Our conjecture that this is a norm in geometry classrooms when doing proofs neither means that we endorse it as desirable or correct, nor that we consider it as determining what will inexorably happen. As Herbst and Chazan (2012) have noted, norms are expectations that participants of a situation have of their behavior in those situations, but they need not be shared by nonparticipant others (for example, mathematicians) who may look at the situation from a different perspective (e.g., may not even recognize the situation). Also norms can be negotiated, even flouted, by participants in practice. Our interest in norms is to investigate the possibilities for norms to be negotiated in ways that increase the quality of students’ mathematical work, while maintaining the feasibility of the work of teaching and honoring the whole of the professional responsibility of mathematics teachers. For example, Cirillo and Herbst (2012) have argued for the value of proof tasks in which students figure out what givens are needed in order to prove a conclusion or where they deduce conclusions from a set of givens. Such activities represent better than usual proof exercises the mathematical experience of proving (Lakatos, 1976). But, how reasonable is it to expect teachers to assign such proof tasks? Hence our interest in exploring the likelihood and justifiability of decisions that stray from the norm.

In order to test whether teachers recognize the hypothesized norms and obligations and to investigate the extent to which they would make decisions consistent with those norms or obligations, we have developed scenario-based assessments (Weekley & Polyhart, 2013). Scenario-based assessments present a narrative of a professional scenario and ask the participant to make a decision at a crucial point. While these narratives can be presented in written form, building on earlier work with animations of classroom interaction (Chazan & Herbst, 2012), our instruments take the form of storyboards populated with nondescript cartoon characters. Multimedia assessments of this sort have been argued to have greater face validity than those that rely on text alone (Olson-Buchanan & Drasgow, 2006). Furthermore, because language alone is a fairly limited set of resources for representing action, multimedia questionnaires that include multimodal representations of possible actions are likely to have more content validity than written questionnaires (see Herbst & Chazan, 2015). We disseminate these assessments online and use the scenarios as part of a suite of assessments that include open-ended and close-ended questions regarding the content of the scenarios and the depicted teacher’s actions.

4.1 Instruments that assess participant recognition of norms

We first designed a set of scenario-based instruments to assess teachers’ recognition of norms of instructional situations using a virtual version of the ethnomethodological breaching experiment (Herbst & Chazan, 2015). We use a customizable graphic language to create storyboards that represent scenarios of teaching (Herbst, Chazan, Chen, Chieu, & Weiss, 2011). The storyboards are used as probes in a multimedia questionnaire that includes several item sets, each based on a storyboard, that are delivered online (using the LessonSketch Footnote 5 platform).

For each item set, participants view a storyboard and then answer a series of open- and closed-ended questions that refer to that storyboard. The participant first responds to a general prompt—“What did you see happening in this scenario?”—and then provides answers to two close-ended questions: “How appropriate was the teacher’s facilitation of work in this scenario?” and then a question that is a bit more specific about the situation being enacted. In the case of instruments that gauge recognition of norms for “doing proofs”, the question is, “How appropriate was the way in which the teacher handled setting up the proof problem for the class?” For the rating questions, participants answer using a six-valued Likert-like response format of appropriateness, with choices that range from (1) very inappropriate to (6) very appropriate. Participants also respond to open-ended follow up prompts—“Please explain your rating”—for both rating questions. The purpose of the open-ended questions is to provide an opportunity for researchers to observe whether participants notice the compliance with,  or breach of, the norms under study.

The storyboards created to investigate the given-prove norm of the doing proofs situation include scenarios where a geometry teacher departs from the hypothesized norm: For example the teacher may provide the statements that are given but ask the class what they might want to prove. We refer to these as the breach storyboards. The questionnaire also includes scenarios designed to represent ordinary instances of the doing proofs situation, during which the teacher complies with the hypothesized norm by providing both the ‘given’ and ‘prove’ statements for the students. We refer to these storyboards as control storyboards.

An example of how storyboards show a teacher’s departure from the hypothesized norm is provided by the storyboard “A proof about a parallelogram”. In this storyboard the teacher begins by drawing a diagram of a parallelogram with points that could be midpoints and segments that could be midpoint segments connecting them. The teacher does not provide any givens that declare the properties of the figure represented by the diagram. Rather, the teacher asks the students in the class to generate a list of properties that could be true of the figure. The students contribute a number of these and the teacher marks them in the diagram that is drawn on the board. Next, the teacher asks the students to identify a property about the figure that they could prove. The scenario concludes with the teacher asking students to work in pairs and identify the “smallest number of givens” that they could use to prove the “strongest claim”. In this scenario, the teacher breaches the hypothesized norm by having students assume the responsibility for determining the statements that will be given and the claim that will be proved. In asking participants what they see happening in the scenario, we expect to read descriptions that allude to the fact that the teacher did not provide the ‘givens’ and the ‘prove.’

After viewing each storyboard, participants answer the open- and closed-ended questions described above. The open responses are independently analyzed by two coders using a norm-recognition coding scheme. The purpose of the scheme is to determine whether a response mentioned that the teacher in the storyboard failed to provide either the statements to be given or the claim to be proved. A response that indicated that the teacher in the scenario did not provide either the statements to be given or the claim to be proved is coded as a “yes” for “response recognizes the breach of the norm”. For each of the breach storyboards, a participant is assigned a score of 1 if there is any open response in the item set coded as “yes” for “recognizes breach of the norm” and 0 otherwise. The sum of these scores across the item sets is used to define a composite norm recognition score (see Herbst, Aaron, Dimmel, & Erickson, 2013, for details). Thus the array of item sets that are each centered on a particular storyboard can be used to determine whether participants recognize the existence of a norm.

4.2 Instruments that assess participant recognition of obligations

To complement the norm recognition instruments, we have developed instruments that attempt to measure teachers’ recognition of the four professional obligations. To gauge the extent to which teachers recognize the obligations as a resource in justifying decisions in teaching, we have designed a suite of four scenario-based instruments, one for each of the four obligations. Each instrument consists of some 15–18 item sets, each of which includes a storyboard, a closed-ended rating question, and an optional open-response field where participants can comment on their ratings. The storyboards represent episodes in teaching in which a teacher deviates from what we hypothesize is normativeFootnote 6 in those circumstances; instead, the teacher does something that presumably attends to one of the four professional obligations. Following the storyboard, participants are asked to rate the degree to which they agree with a statement, using a six-valued Likert-like response format for agreement—(1) strongly disagree to (6) strongly agree. The statements that participants are asked to agree or disagree with are scenario-specific, but they all have the same structure: “The teacher should have [taken (a normative) action] rather than (summary of the action the teacher has taken in the storyboard)”.

The statements that participants rate are thus structured to gauge the extent to which participants judge as justifiable the departures from the normative courses of action that are depicted in the storyboards. In our account of practical rationality we expect that the four professional obligations act as resources that teachers could use to justify departures from what is normative. The scenario-based obligations items are designed to test this aspect of the theory. As part of our instrument development process we conducted focus-group reviews of the item sets for each instrument to help ensure that (1) the teaching actions represented in each storyboard are recognizable as departures from what is normative and that (2) the departures could be justified by the professional obligation that was designed to be at play in the storyboard. The participants in these focus groups have been experienced teachers of mathematics, mathematics educators, and mathematicians.

An example of an item set from the disciplinary obligation instrument helps illustrate the design of these obligations items. The item set includes a storyboard consisting of three-frames, during which a teacher is shown modifying a definition that has been provided in the textbook. The teacher states that the reason for the modification is to ensure that what is defined is a function (as opposed to a relation). That mathematics teachers might choose to modify the way that a mathematical idea is presented in a textbook seems justifiable on account of their obligation to the discipline of mathematics. At the same time, it would be normative for a teacher to use the definition exactly as it is provided in the textbook the class is using. The items are designed to probe the tension between doing what might normally be done and deviating from the norm on account of a professional obligation.

After viewing the storyboard where a teacher is shown modifying a definition, participants are asked to rate the degree to which they agree with the following statement: “The teacher should keep to what is in the textbook, rather than require students to use information that is different from what is in the textbook”. The statement is designed to state an alternative, normative action that a teacher in this situation could take—in this case: using the definition as it is provided in the textbook—and pit this against the action that is depicted in the storyboard. Our hypothesis is that a mathematics teacher who recognizes the obligation to the discipline of mathematics would disagree with this statement. We take such disagreements as evidence that the teacher’s departure from a normative course of action is reasonable in this situation. The conclusion that the action is deemed reasonable on account of the disciplinary (or the other target) obligation follows from the design of the storyboards and the use of focus groups to vet them.

4.3 Toward an instrument that records teachers’ decisions in situations

We have also been developing an instrument to record decisions teachers make when confronted with the instructional situations of interest.Footnote 7 The decision instrument includes items that present participants with the need to choose what to do in the context of an enactment of the target situation. As a rule, the participant is ushered into the scenario by way of a stem-story that identifies the goal of the work the class will do—for the case of the given-prove norm, the stem describes the goal as having students do a proof. The stem-story provides some mathematical context for what the students’ work will be on and the scenario stops at a moment when the teacher would be expected to act. For the case of the given-prove norm, the stem-story stops before the teacher has either stated the full problem or given some of that responsibility to students. Figure 1 shows a stem-story in which the teacher has just reviewed the definition of an isosceles trapezoid and proceeds to draw a diagram on the board along with givens and a statement to prove. A student interjects at this point in order to suggest another given. Thus the teacher must make a decision about how to react to the student’s contribution.

Fig. 1
figure 1

Stem-story for a decision item from the first and second generation: a the teacher asks and receives a definition from a student, b affirms the student’s contribution, c draws and labels a diagram on the board, and d writes a proof problem when a student suggests an additional given

Full size image

Our decision instrument is made of four item sets for each norm and contextualized in an enactment of the instructional situation in which the norm belongs. Each item set consists of a set of questions posed about a scenario like the one shown in the Fig. 1. The first two questions (“What action would you do next?” and “Please explain your reasoning for this answer.”) ask the participants to describe in an open response box what they would do next and to justify their response. Then the participant is offered four choices of what they could possibly do. These choices are presented in four distinct one-frame storyboards. Participants are asked to choose one of these four storyboards as the action they would do if they were in the situation. In designing this instrument we have considered a number of options for how to create the choices. We offer all of them here because they are all reasonable ones and their diversity helps us illustrate the need to consider the basis upon which the item set is designed.

Our first generation decision items considered the various choice options as dependent on an analysis of the specifics of the story, including the mathematics of the problem. We created one option in which the teacher follows the norm of the situation; the other three options are breaches of the norm that might be justified based on some feature of the story. This format can be seen in Fig. 2 below where we display the four options we created for the stem-story shown in Fig. 1. In the first option, the teacher does not accept the student’s contribution; in the second option, the teacher asks the student to explain how the additional given would help; in the third option, the teacher puts the inclusion of the additional given to a vote; in the fourth option, the teacher accepts the student’s additional given without further remark.

Fig. 2
figure 2

The four options for the item shown in Fig. 1

Full size image

Note that since breaches were closely related to the content of the stem-story, the different item sets were not necessarily comparable in regard to the array of choices offered. This had consequences for the analysis. In particular, as Kosko and Herbst (2012) showed, one could examine the extent to which participants departed from the conjectured norm within a depicted scenario. Alternatively, such differences in the context of each scenario might allow one to compare differences between responses to scenarios if the possible decisions are similar enough (Kosko, 2013). Still the first generation of the instrument allowed us to collect pilot data and investigate an initial regression model where both MKT and norm recognition were included as predictors.

The second generation of the decision instrument was designed to have choices that were comparable across items, as well as consistent with the storyline provided in the story stem. In particular we chose to create alternatives to the norm that could be justified as providing opportunities for different types of student participation. For example Fig. 3 shows the alternatives (i.e., response options) we created for the item in Fig. 1. The first option follows the norm in much the same way as the original version of the item; the second option has the teacher asking the students to work with their partners to see if they can prove anything stronger with the additional given; the third option has the teacher soliciting contributions from the rest of the class; the fourth option has the teacher encouraging the students to decide for themselves whether they want to use the additional given. As illustrated by this example, the scenario for each item set had a normative option, an option where the students determine the ‘given’ and ‘prove’ individually, an option where students talk to their partners and decide privately what the ‘given’ and ‘prove’ should be, and an option where the whole class discuss the ‘given’ and ‘prove.’

Fig. 3
figure 3

The four options for the second generation of the item whose stem story is shown in Fig. 1

Full size image

Using this revision of the items, it was possible to consider not only scoring all items in such a way that we could tell the extent to which participants made a normative choice,Footnote 8 but also the extent to which participants allowed for greater student participation in the discussion.Footnote 9 As each item had one normative option; when we administered these items to a convenience sample of 42 high school teachers we were still able to code each participant’s response dichotomously depending on whether they chose the normative option or not. Coded this way, the four decision items had a Cronbach’s Alpha of 0.431 and when they are used to determine how often teachers choose the normative option, a sample of high school mathematics teachers had a mean of 0.74 for the sum of all four items (where the minimum possible score was 0 and the maximum possible score was 4) and a standard deviation of 0.939 (N = 42). Because all the items were created in such a way that two of the alternatives increased students’ opportunity to talk and two alternatives (including the normative choice) decreased such opportunity, we could also code items to create a measure of the extent to which participants would encourage students’ engagement in talk, by creating a dichotomous variable (Max_Talk) that took the value 1 if a participant chose either the option that encouraged the students to work in pairs or the option to have a class discussion and 0 otherwise. Scored this way the sample had mean of 2.90 (where the minimum possible score was 0 and the maximum possible score was 4) and a standard deviation of 0.983 (N = 42) but the items had lower internal consistency (Cronbach’s Alpha of 0.276).

Given our argument that professional obligations are used to justify departures from the norm, we sought to redesign the instrument in a way that would let us investigate how the four professional obligations were entering into teacher’s decisions about what action to take in the scenarios. We also noticed that the connection between such meaning and the professional obligations of mathematics teaching was rather muted. We decided to further redesign the items to allow them to detect departures from a norm that were justifiable on account of particular obligations. The current, redesigned items match each of the four decision items for a given norm to one of the four obligations. For each item, participants might decide to choose the normative option or might depart slightly, moderately, or radically from the norm, with each of those departures justifiable, in principle, according to the target obligation. For example, Fig. 4 shows the story stem for a decision item that is a revision of the story-stem shown in Fig. 1. In the revised story stem the teacher is speaking one-on-one with a student when the student proposes modifying the givens for the problem. This change was made in order to target the teacher’s obligation to the individual.

Fig. 4
figure 4

Stem-story for a third generation item set derived from that shown in Fig. 1

Full size image

Figure 5 shows the alternatives associated with the story stem from Fig. 4. The first option complies with the norm (i.e., the teacher does not accept the student suggestion to omit one of the givens) while each subsequent option is increasingly distant from the norm: In one option the teacher lets the student do the proof without the given in question; in a second option the teacher suggests that the student might modify the problem and end up choosing a different ‘prove’; and in a third option the teacher departs from the situation of doing proofs entirely by giving the student the opportunity to formulate a different geometry problem. The stem of the item, where the teacher recognizes that the student who made the suggestion is “comfortable with these proof problems” is what positions each of the options as a response to the obligation to the individual.

Fig. 5
figure 5

The four options for the third generation item whose stem story is shown in Fig. 1

Full size image

Those items can be scored 0–3 depending on whether the participant chooses the normative option (0) or the degree to which they depart from the norm (1, minor departure; 2, moderate departure; 3, major departure). Since the four decision items for each of the four norms we are investigating can be classified as associated primarily with one norm and one obligation, they can be aggregated to form different scores. Two scores of interest are (1) the extent to which the respondent chooses to abide by a norm (aggregating scores across items associated with a given norm) instead of departing from that norm, and (2) the extent to which a respondent recognizes a given obligation as a warrant to depart from a norm (aggregating scores across items that offer departures from various norms on account of a given obligation). In the first case we would argue that the closer participants’ scores are to 0, the more the participants choose decisions closer to the norm in a given instructional situation (regardless of the departures that are available). In the second case—and for each obligation—we would argue that the farther participants’ scores are from 0, the more that participants gravitate toward honoring that obligation, regardless of the norm they confront.

5 Examples of analyses that can be done with such measures

In this section, we discuss how data from these instruments can be analyzed to explore the hypothesis that the actions of mathematics teachers (as assessed in the decision instrument) can be justified by various contextual resources. This discussion provides examples of how such analysis can be conducted, but it is not prescriptive.

A teacher’s decision to depart, or not, from an instructional norm can be considered either in regards to particular scenarios or, more generally, across scenarios, with both considerations depending upon the characteristics of the scenario in which a decision is to be made. It is tempting to consider the general trend as an ultimate goal in understanding common factors that affect or influence teachers’ instructional decisions across instances of the same instructional situation. However, our own study of such contexts suggests there is similar value in examining effects associated with decisions made in particular instances of instructional situations that may be shared by multiple teachers. To examine decision making in both particular scenarios and the general trends across scenarios, we have conceptualized approaches using nonparametric and parametric regression models to examine the nominal nature of decisions as variables (e.g., Kosko, 2013; Kosko & Herbst, 2012). In this section, we discuss how we have analyzed data from the instruments we have described along with future directions for analysis.

Kosko and Herbst (2012) provide an example of how to examine the decisions made within a particular scenario using the first generation of decision items assessing the norm that the geometry teacher is the one who provides the givens and prove statement in the posing of a proof problem. Within the examined scenario, the depicted geometry teacher was reviewing with students what was needed to write a proof; particularly that a set of ‘givens’ and a ‘prove’ statement were needed to begin such work. Participating teachers were then provided with four options. The first three (Choices A–C) represented different ways in which the norm (that the teacher is the one who provides the givens and prove statement when posing a proof problem) could be breached. The last option (Choice D) was compliant with the norm. The choices made by our pilot sample of 55 secondary mathematics teachers showed a roughly even distribution in choices (Choice A = 22.7 %; Choice B = 18.2 %; Choice C = 31.8 %; Choice D = 27.3 %), seemingly indicating that the choice of teaching decision was random. Yet, as we have suggested previously, various factors can and do influence teachers’ decision-making, including resources attributable to the individual and those attributable to the context (in this case, that the depicted scenario is an instance of the situation “doing proofs”). To help make sense of why teachers chose certain decisions over others, Kosko and Herbst (2012) used multinomial logistic regression (MLR).

MLR is a nonparametric analysis that allows for examining nominal variables as the outcome measure in a regression equation (see Hosmer & Lemeshow, 2000, for a detailed description). In the context of examining teachers’ choice of decision in particular scenarios, one potential decision is considered in comparison with the others. Key in our application of MLR with the first generation of decision items is the designation of the different decisions as nominal; that while one decision is considered to endorse the norm, the other decisions are equally plausible dependent upon the justification a teacher may provide in taking such action.Footnote 10 Kosko and Herbst (2012) compared the probability of participating teachers selecting each of the alternatives to the norm against the probability that they selected the teaching decision that complied with the norm. Resources attributable to the individual [teachers’ perceived pedagogical autonomy, years of experience, mathematical knowledge for teaching geometry (MKT-G) score] and to the situation (i.e., an indicator of teachers’ general recognition of the norm at play, assessed using a norm recognition instrument) were included as variables in the regression model (see Eq. 1). Kosko and Herbst’s (2012) analysis did not include variables for various professional obligations or other individual resources. Such inclusion would require additional sample size and potentially different analysis models (described below).

$$\begin{aligned} g_{m} (x) & = \ln \left[ {\frac{P(Y = m|x)}{P(Y = 0|x)}} \right] \\ & = \beta_{m0} + \beta_{m1} (normativity)_{1} + \beta_{m1} (YearsExperience)_{2} \\ &\quad+ \beta_{m3} (MKT - G)_{3} \\ \end{aligned}$$
(1)

The methodological lesson that can be taken from Kosko and Herbst’s (2012) analysis is that for certain comparisons of decision options, different factors included as predictors affected the probability of choosing certain decisions over others. For example, the more years of experience a participant had, the lower the likelihood they would choose Choice A over Choice D, but this did not have a significant effect on the probability of choosing either of the other two decisions over Choice D (the action compliant with the norm). However, the degree to which a teacher generally recognized the norm tended to affect the probabilities of all comparisons. This illustrates specifically how individual resources may or may not influence the decisions made in a particular scenario; the data lends credence to the assertion that recognition of the characteristics of the particular instructional situation do matter. This application of MLR to examining decision making allows one to tease out how particular resources do or do not affect decisions within a specific scenario.

As thus far described, MLR allows for examining the effect of factors on teachers’ decision-making in particular teaching scenarios. However, MLR can also be used to compare the influence of such factors across different teaching scenarios. Kosko (2013) examined three scenarios in order to compare when elementary teachers chose probing questions in mathematical discussions. Using a similar approach to Kosko and Herbst (2012), Kosko (2013) included an MKT score as an individual resource of participants. However, the context of one scenario varied from the other two, allowing for a comparison of how certain variations of a context can affect the influence of various factors on decisions, such as particular teaching norms or individual resources. Figure 6 illustrates that while Scenarios 1 and 3 in Kosko’s (2013) analysis yielded similar relationships between MKT scores and likelihood of choosing an action that endorsed the norm that asking a probing prompt is appropriate, this trend is substantially different for Scenario 2. Therefore, examination of similar decisions across different scenarios can illustrate how the instructional context of a scenario affects decision making.

Fig. 6
figure 6

Kosko’s (2013) comparison across three scenarios

Full size image

MLR can be used to examine both how factors affect the choice of one decision over another and how variation due to context in different scenarios affects how such factors influence these choices. These applications of MLR allow for finer-grained examinations of decision making in particular scenarios. Comparison of specific decisions within a single scenario, as illustrated by Kosko and Herbst (2012), allows for examination of how factors influence the choice of one decision over another. However, comparisons across a small set of scenarios, as illustrated by Kosko (2013), allows for examination of how such factors’ influence on decisions can change given variations in the contexts of scenarios.

One reason for the revisions associated with the decision items previously described was to allow for modeling of the general nature of decision making as associated with particular scenarios. Indeed, the finer-grained approaches incorporating MLR can be considered as precursors to regression models that include examination of multiple scenarios, as represented by composite scores. In the third generation of decision items, a score associated with norm adherence (or departure) is provided based on participating teachers’ responses. Therefore, across multiple items for a given professional obligation, a score can be calculated representing the average degree of departure from a norm. A parametric multiple regression equation can then be modeled to examine the effect of different factors on teachers’ general tendency to adhere or depart from a particular instructional norm. Equation 2 provides an illustration for one possible model given these circumstances.

$$\begin{aligned} Y & = \beta_{0} + \beta_{1} (MKT - G) + \beta_{2} (YearsExperience) + \beta_{3} (\text{Re} cognizeNorm) \\ & \quad + \beta_{4} (\text{Re} cognizeObligation) + e \\ \end{aligned}$$
(2)

In Eq. 2, the outcome represents a teachers’ tendency to depart from a specific instructional norm. Their MKT-G score, years of teaching geometry, as well as scores representing their general recognition of the norm and obligation at hand are included as predictor variables. Results from modeling decision making with this approach allows for conjectures regarding how and why teachers go about making certain decisions in the general case. Important in this consideration is that while regression models such as those displayed in Eq. 2 allow for some generalization about decision making, there are always exceptions to generalizations. In such cases, it is useful to revisit more fine-grained examinations that incorporate MLR, or conduct more qualitative analysis for a deeper examination of the phenomenon at hand.

The parametric and nonparametric regression analyses just described can be further extended to consider effects of the nested structure of the educational system (or other social structures). Equations 3 and 4 provide an example of how such comparisons can be modeled using hierarchical linear modeling (HLM; see Raudenbush & Bryk, 2002). Extending the previous regression example (see Eq. 2), one can consider teachers’ decision making as being influenced by different factors related to their school environment. These can include, but are not limited to, whether the school is a magnet for math and science, the average MKT-G score of mathematics teachers employed in the school, or the average score for recognition of each of the obligations among teachers in a school. Equations 3 and 4 provide an example with such factors included. In this particular example, various factors at the school level can be included at level-2 to detect their influence on teachers’ tendency to depart from an instructional norm, given the potential for deviating from the norm on account of the disciplinary obligation. This particular example models the interaction between school and teacher level variables on only two effects, but other variations can be conceptualized depending on the nature of decisions being modeled, or one’s particular focus in modeling them. Further, one can adjust the application of this model to determine the degree to which context (such as school type) affects various resources (i.e., recognition of the institutional obligation, or teaching norms).

Level-1:

$$\begin{aligned} Y_{ij} & = \beta_{0j} + \beta_{1} (MKT - G) + \beta_{2j} (YearsExperience) + \beta_{3j} (\text{Re} cognizeNorm) \\ & \quad + \beta_{4j} (\text{Re} cognizeObligation) + \mu_{ij} \\ \end{aligned}$$
(3)

Level-2:

$$\begin{aligned} \beta_{3j} & = \gamma_{00} + \gamma_{01} (Mean\_MKT - G)_{j} + \gamma_{02} (Curriculum)_{j} + \gamma_{03} (Mean\_SES)_{j} \\ & \quad + \gamma_{04} (CurriMean_{{Disciplinary_{Obligation} }} )_{j} + u_{0j} \\ \beta_{3j} & = \gamma_{30} + \gamma_{31} (Mean\_MKT - G)_{j} + \gamma_{32} (Curriculum)_{j} + \gamma_{33} (Mean\_SES)_{j} \\ & \quad + \gamma_{34} (CurriMean_{{Disciplinary_{Obligation} }} )_{j} + u_{3j} . \\ \end{aligned}$$
(4)

The benefit of using HLM to model teachers’ decision making with the multimedia questionnaires we have thus far described is that such an approach allows for inclusion of various associated factors beyond those that describe the individual teacher. As we have described earlier, when teachers consider a course of action among various alternatives in a particular scenario, they bring resources attributable to the individual as well as the context in which they are nested. This context can be the school, as shown, but it can also be a professional subclass the teacher belongs in (e.g., experienced teachers of geometry, which might show in different average recognition of an instructional norm in geometry). The nature of the multimedia decision questionnaires allows for eliciting answers from numerous teachers that still pay attention to context. Our framework provides means for operationalizing certain resources related to this context (i.e., individual’s recognition of instructional norms or professional obligations; see Chazan et al., in press). These resources can then be modeled to examine how they affect decisions in particular scenarios (via MLR) or to examine more general relationships regarding teachers’ tendencies to make certain decisions across numerous scenarios of a certain type (via multiple regression). While a teacher’s recognition of certain norms or professional obligations, as well as indicators of their knowledge and beliefs, can provide useful information regarding why teachers make certain decisions in teaching scenarios, HLM allows for inclusion of factors that are associated with the various social contexts in which teachers are nested. Our own descriptions of the importance of context within decision making (as described in the case of our decision item scenarios) suggests that the context in which decisions are made, and the various resources teachers bring to the decision making (both individual and contextual), are important to consider. However, the data demands of HLM are quite large and not often easily obtained by researchers. For example, the model illustrated in Eqs. 3 and 4 could be examined with a sample of 310 teachers across 31 schools, whereas the previously described regression models would require approximately one-seventh the sample size.Footnote 11

6 Conclusion

When considering teacher decision making in the context of curriculum implementation or use of curriculum resources, mathematics education researchers have used conceptualizations such as teachers’ beliefs, mathematical knowledge for teaching, and goals (Schoenfeld, 2011). We have argued for going beyond individual factors in the study of teachers’ decision making by considering other resources that pull teachers toward both reproducing and changing their practice. In particular we brought attention to instructional norms and professional obligations as two sets of contextual information resources that might help account for decisions teachers make.

In this paper, we have illustrated possible ways in which teachers’ recognition of characteristics of instructional norms and of characteristics of their professional position (e.g., the professional obligations) could be measured. We also have illustrated also how a particular kind of curricular decision making could be simulated and captured: How multimedia questionnaires can be used to elicit teachers’ decisions in instances of instructional situations where a norm might be active. Finally, we’ve shown how responses to those instruments can be put together with measures of individual factors (such as MKT, beliefs, or teachers’ experience) into regression models. Such models can help understand whether and how individual decisions might be reducible to individual factors such as knowledge or belief or also depend on matters associated with institutional context.

The approach to studying teacher decision making that we have thus outlined can contribute to the literature on teacher decision making in several ways. First, by assessing teachers’ recognition of norms and obligations through their responses to scenarios, we avoid the assumption that the rationality behind action is explicit and available to individuals. This helps move the study of teacher decision making beyond the model of the teacher as a rational actor. Second, by providing measures of those two constructs, as well as of teacher decision making, and analytic techniques that allow researchers to study the impact of these constructs on teacher decisions, we show how dimensions of social context can be brought into the study of teacher decision-making.

While the success of the approach articulated here will eventually depend on the analysis of data being currently collected, we argue that researchers on instruction can benefit from considering this approach as they think about eliciting and accounting for instructional decision making. Specifically, by nesting decision making in instructional situations, researchers can take advantage of observed (or hypothesized) norms of a situation, as well as anticipate possible departures from the norm that could be justified by recognition of obligations, to build decision making problems that are meaningful for practitioners and informed by current theoretical approaches for the study of instruction.