Keywords

In the United States at this time several characterizations of high quality mathematics teaching are receiving attention from mathematics educators and public policy professionals. Each has at its core one or more important facets of teaching proficiency. Typically these different characterizations are treated in isolation from each other, emphasizing the distinctions between and among them rather than the ways in which they might interact with each other. In this chapter we focus on one view of high quality mathematics teaching that has garnered considerable attention from the education policy community in the United States. We report the results of an analysis in which we probed empirically the extent to which samples of teaching practice associated with that view of highly accomplished mathematics teaching also exhibited characteristic features associated with two alternative views of high quality mathematics teaching: (a) the effective use of cognitively demanding mathematics tasks and (b) the use of progressive pedagogical practices. We performed our empirical analysis on samples of instructional activity drawn from actual mathematics classrooms—samples that were selected by teachers as examples of their “best practice.”

Characterizations of High Quality Mathematics Teaching

In this section we discuss the three views of high quality mathematics teachers and teaching that we consider in the study reported here. We begin with the notion of highly accomplished (mathematics) teachers and teaching proposed by the National Board for Professional Teaching Standards (NBPTS). The approach taken by the NBPTS was intended to characterize high quality teachers and teaching in a generic way, and then to develop specific characterizations for several school subjects, including mathematics. In the study reported here we begin with the NBPTS view of highly accomplished teaching and teachers and examine the extent to which this view is consistent with two other characterizations of high quality mathematics teaching that derive from research in the field of mathematics education. In contrast to the NBPTS approach, which considers first the features of high quality teaching in general and then tries to specify particular versions for subject matter teachers and teaching, the latter views we consider are derived from research that specifically examined teachers and teaching in mathematics classrooms. In this section we also describe these two alternative perspectives on high quality mathematics teaching.

Highly Accomplished Teaching: NBPTS Certification

As one means of improving the teaching profession in the United States, the NBPTS was established in 1987 to recognize highly accomplished teachers by delineating what high quality practice looks like and then devising a way to identify those who exhibit it. To accomplish its goal, the NBPTS used professional consensus to establish standards for what accomplished teachers should know and be able to do, after which it developed a national voluntary system to assess and certify teachers who meet the standards. Thus, in this view, high quality teaching is what NBPTS certified teachers do in their classrooms.

The NBPTS recognizes accomplished practice in a number of fields. Except for generalist certifications, each field is defined by content area (e.g., mathematics) and students’ development level (e.g., Middle Childhood-Early Adolescence, ages 7–16). The NBPTS certification system began with the specification of standards for professional practice, initially at a very broad general level, and then for each content-area/age-level certification field. Figure 1 displays the 12 standards, distributed across four broad areas of competence, along with some sample elaborations, for Early Adolescence/Mathematics (EA/M), which is the certification field we studied (see http://www.nbpts.org/for_candidates/certificate_areas1?ID=8&x=42&y=8 for more information regarding current NBPTS certification areas).

Fig. 1
figure 4_1_162454_1_En

NBPTS standards for early adolescence/mathematics (adapted from NBPTS, 1998, pp. 11–12)

Full size image

Applicants for NBPTS certification complete a series of assessment tasks in which they are asked to demonstrate knowledge and professional practice of many kinds, and their overall performance determines whether they receive NBPTS recognition. Each component of the assessment is linked to one or more of the standards for the certification area. The EA/M assessment consists of two parts: in one, teachers complete an on-demand, test-center-administered set of exercises to evaluate certain aspects of their content and pedagogical content knowledge; in the other, candidates submit a portfolio that includes contextualized samples of their teaching practice and reflections on their work. For applicants in 1998–1999, which is the data set examined in this study, the portfolio component of the EA/M assessment consisted of six entries, of which four were classroom–based entries. The two portfolio entries (Developing Mathematical Understanding and Assessing Mathematical Understanding) examined in this chapter captured teaching practice via classroom artifacts, samples of student work, and teachers’ reflective narratives.

The NBPTS assessment process has been extensively evaluated. Technical analyses of the reliability and validity of the assessment have been conducted (e.g., Bond, Smith, Baker, & Hattie, 2000), and there have been a number of studies investigating the relationship between NBPTS certification and measures of teaching practice and teacher effectiveness, especially in regard to student achievement (e.g., Hakel, Koenig, & Elliott, 2008). In general, the research points to a strong positive relationship between NBPTS certification and student achievement; that is, students of teachers who have attained NBPTS certification tend to perform well on standardized achievement measures.

Effective Use of Cognitively Demanding Mathematics Tasks

An alternative view of high quality mathematics teaching considered in this study is one derived from research on classroom mathematics instruction in the United States and elsewhere. International surveys of the mathematics achievement of students around the world regularly indicate that the average performance of students in the US is mediocre when compared to that of students in many other countries, especially countries in Asia (e.g., Lemke et al., 2004; Mullis, Martin, Gonzalez, & Chrostowski, 2004). A recent analysis of the performance of students in 12 countries who participated in both TIMSS and PISA found that students in the United States have specific weakness in using high-level cognitive processes, such as reasoning and problem solving (Ginsburg, Cooke, Leinwand, & Pollock, 2005).

It is quite likely that the student deficiencies in using high-level cognitive processes on mathematics test items are largely a consequence of the limited opportunities they have to learn mathematics in classroom lessons. Mathematics classroom instruction is generally organized around and delivered through mathematical tasks, activities, and problems. According to Doyle (1983, p. 161), “tasks influence learners by directing their attention to particular aspects of content and by specifying ways of processing information.” In fact, tasks with which students engage constitute, to a great extent, the domain of students’ opportunities to learn mathematics. Students in all seven countries analyzed in the TIMSS Video Study (NCES, 2003) spent over 80% of their time in mathematics class working on mathematical tasks.

Tasks can vary not only with respect to the mathematics content but also with respect to the cognitive processes that they entail. Tasks that require students to analyze mathematics concepts or to solve complex problems offer opportunities for students to sharpen their thinking and reasoning in mathematics. In contrast, tasks that require little more than memorization and repetition offer less opportunity to develop proficiency with high-level cognitive processes. Moreover, the cognitive demands of mathematical tasks can change as tasks are introduced to students and/or as tasks are enacted during instruction (Stein, Grover, & Henningsen, 1996). The Mathematical Tasks Framework (MTF) [see Fig. 2], models the progression of mathematical tasks from their original form to the tasks that teachers actually provide to students and then to the tasks as they are enacted by the teacher and students in classroom lessons.

Fig. 2
figure 4_2_162454_1_En

Mathematical tasks framework (adapted from Stein, Smith, Henningsen, & Silver, 2009, p. xviii)

Full size image

The tasks, especially as enacted, have consequences for student learning of mathematics. The leftmost two arrows in Fig. 2 identify critical phases in the instructional life of tasks at which cognitive demands are susceptible to being altered.

In the TIMSS 1999 video study, the ability to maintain the high-level demands of cognitively challenging tasks during instruction was the central feature that distinguished classroom teaching in countries where students exhibited high levels of mathematics performance when compared with countries like the United States, where performance was lower and teachers rarely maintained the cognitive demands of tasks during instruction (NCES, 2003; Stigler & Hiebert, 2004; Hiebert et al., 2005). In that study, a random sample of 100 eighth-grade mathematics classes in each of seven countries was videotaped during the 1999–2000 school year. Although 17% of the tasks used by teachers in the United States were coded as high level, none was implemented as intended. Instead, most “making-connections” problems were transformed into procedural exercises. The authors concluded that 8th grade students in the United States spent most of their time in mathematics classrooms practicing procedures regardless of the nature of the tasks they were given. This claim is consistent with an analysis of mathematics instruction conducted by the Horizon Research Institute, in which only 15% of observed mathematics lessons were classified as providing opportunities for complex thinking, or for mathematical reasoning or sense-making (Weiss & Pasley, 2004; Weiss, Pasley, Smith, Banilower, & Heck, 2003).

Beyond the research documenting modal practice in US. classrooms some other research recently conducted in a variety of American classroom contexts has found that student learning does occur if cognitively demanding mathematical tasks are used regularly and if the high-level cognitive demands are consistently maintained in classroom lessons (Boaler & Staples, 2008; Hiebert & Wearne, 1993; Stigler & Hiebert, 2004; Stein & Lane, 1996; Tarr et al., 2008). For example, in a longitudinal comparison of three high schools over a 5-year period, Boaler and Staples (2008) determined that the highest student achievement occurred at the school in which students were supported to engage in high-level thinking and reasoning. Boaler and Staples attribute students’ success to the teachers’ ability to maintain high-level cognitive demands during instruction, especially the teachers’ use of pre-planned questions that elicited and supported students’ thinking. Studies by Tarr and colleagues (2008) and by Stein et al. (1996) both found that classrooms in which teachers consistently encourage students to use multiple strategies to solve problems and support students to make conjectures and explain their reasoning were associated with higher student performance on measures of thinking, reasoning, and problem solving.

Emerging from this array of theoretical and empirical work in and on mathematics classrooms is a view of high quality mathematics teaching in which teachers regularly provide students with worthwhile and challenging tasks and generally maintain the level of cognitive demand as students engage with the tasks in a lesson. Thus, this view of high quality mathematics teaching is different from the NBPTS characterization both in kind and in origin. Next we describe a third view that also derives from research in mathematic classrooms, but that is different in kind from both the NBPTS and cognitive demand characterizations.

Innovative Pedagogy

Another alternative view of high quality mathematics teaching encompasses a set of instructional practices that are generally thought to represent progressive ideas about mathematics teaching and that have been associated in various ways with teaching mathematics for understanding. As noted earlier, research in mathematics classrooms in the United States in the upper elementary and middle school grades has found that classroom instruction typically eschews the use of technological tools or concrete models for abstract ideas, tends to focus tasks that make little or no connection to the world outside of school, and pays little or no attention to the development of meaning (e.g., Stigler & Hiebert, 1999; Stodolsky, 1988). Such pedagogy is at odds with current conceptualizations of how people learn best when the goal is developing understanding (Bransford, Brown, & Cocking, 1999). Certain innovative pedagogical practices are often associated with the phrase, teaching mathematics for understanding. Over at least the past 60 years a solid body of research evidence has amassed pointing to the benefits of teaching for understanding (sometimes called by various other names, including authentic instruction, ambitious instruction, higher-order instruction, problem-solving instruction, and sense-making instruction) in mathematics (e.g., Brownell & Moser, 1949; Brownell & Sims, 1946; Carpenter, Fennema, & Franke, 1996; Carpenter, Fennema, Peterson, Chiang, & Loef, 1989; Cohen, McLaughlin, & Talbert, 1993; Fuson & Briars, 1990; Hiebert & Wearne, 1993; Hiebert et al., 1996; Newmann & Associates, 1996).

Although there are many unanswered questions about precisely how teaching practices are linked to students’ learning with understanding (see Hiebert & Grouws, 2007), there has been increasing emphasis in the mathematics education community in teaching practices that deviate from the canonical version of classroom mathematics instruction noted above and that appear to be more oriented toward the development of students’ conceptual understanding. Among the hallmarks of this conceptually oriented version of instruction are teaching practices that are suitable to support multi-person collaboration and communication among students, and to engage students with real-world applications or the use of technological tools or physical models (e.g., Fennema & Romberg, 1999; Hiebert & Carpenter, 1992).

Advocates for conceptually oriented teaching in school mathematics (e.g., NCTM, 1989, 2000) have suggested the potential value of fostering communication and interaction among students in mathematics classrooms through the use of complex tasks that are suitable for cooperative group work and that provide settings in which students need to explain and justify their solutions. Moreover, to increase students’ engagement with mathematical tasks and their understanding of concepts, instructional reform efforts have also encouraged the use of hands-on learning activities and technological tools, as well as connecting work done in the mathematics classroom to other subjects and to the world outside school. Beyond exhortations, there is also some research evidence to support these hypotheses about pedagogy that might support students’ development of mathematical understanding (e.g., Boaler, 1998; Fawcett, 1938; Fuson & Briars, 1990; Good, Grouws, & Ebmeier, 1983; Hiebert & Wearne, 1993; Stein et al., 1996). Moreover, there is evidence in some studies that these, and other innovative pedagogical strategies, can be applied in superficial ways that emphasize non-mathematical aspects of the activities and sacrifice the complexity of mathematics content (e.g., Cohen, 1990; Ferrini-Mundy & Schram, 1997; Romagnano, 1994; Schoenfeld, 1988; Weiss et al., 2003; Wilson & Floden, 2001).

Emerging from this array of theoretical and empirical work in and on mathematics classrooms is a view of high quality mathematics teaching in which teachers regularly engage in innovative pedagogical practices; that is, pedagogy that deviates from the canonical portrayal found in research on typical classroom teaching. This view is different from both the NBPTS characterization and the cognitive demand view presented above, and it is the third one considered in the study reported here.

Study Methods

In this study, we examined samples of instructional practice—lesson artifacts and teachers’ commentaries on lessons—submitted by applicants seeking NBPTS certification. The instructional samples were systematically probed for evidence of mathematical and pedagogical features associated with the views of cognitive demand and innovative pedagogy noted above.Footnote 1

Sample

With the cooperation of the NBPTS, we obtained test center and portfolio exercise score data for all candidates (N = 250) who applied for NBPTS EA/M certification in 1998–1999. From this set of 250 applicants we selected a random sample of candidates (n = 32; nearly 13% of the population). Our sample was demographically similar to the entire population of EA/M applicants in 1998–1999 and contained a comparable ratio of successful to unsuccessful applicants to that of the full applicant pool; our sample included 13 individuals who obtained NBPTS certification and 19 who did not.Footnote 2 The awarding of NBPTS certification is based on a composite of weighted scores on 10 performance indicators (six portfolio entries and four test center exercises), each with an independent, though not equal, contribution to an applicant’s overall score.

Data

For each of the 32 individuals in our sample, we obtained copies of the two artifact-based portfolio entries—Developing Mathematical Understanding (DU) and Assessing Mathematical Understanding (AU). These artifact-based entries contained extensive textual portrayals of instructional practice related to developing and assessing student understanding of mathematical ideas, along with supporting artifacts (e.g., students’ work, tests, photographs). The DU entry required two instructional activities, both focused on the same mathematical idea, which could come from consecutive lessons or from nonconsecutive lessons. In contrast, the AU entry required only one activity, and it was required to be different from the idea that was the focus of the DU entry.

Candidates were instructed to provide all of the following information in each portfolio entry: a written description of the instructional context (e.g., grade, subject, class characteristics); a written description of teacher planning (e.g., substantive math idea, goals for instructional sequence, challenges inherent in teaching these activities); analysis of student responses (actual student work samples for these specific students were appended to the entry); and candidate’s reflections on the outcomes of each lesson. For both entries, candidates were instructed to select activities in which students were engaged in thinking and reasoning mathematically (e.g. interactive demonstrations, long term projects, journal assignments, problem solving); they were instructed not to select activities that focused on rote learning (e.g., students’ memorizing procedures).

Data Analysis

Our examination of the NBPTS data consisted of quantitative and qualitative analyses of the two portfolio entries submitted by our sample of 32 applicants. Trained coders examined each entry for evidence of cognitively demanding mathematics tasks and the presence of innovative pedagogical features, and they did so without knowledge of the NBPTS certification status of the applicant whose portfolio entry they were judging.Footnote 3 Following the coding of all portfolio entries with respect to cognitive demand and pedagogical features, we conducted further analyses using these codes to compare the portfolio submissions of applicants who were awarded NBPTS certification with those who were not.

Cognitive Demand of Mathematical Tasks in NBPTS Portfolio Submissions

To assess the cognitive demand character of the mathematical tasks in the portfolio entries we developed coding criteria for high-demand and low-demand activities. Low-demand tasks were those that exclusively involved low-level cognitive processes, such as recalling, remembering, implementing, or applying facts and procedures. In contrast, high-demand tasks were those that required students to use high-level cognitive processes, such as analyzing, creating, evaluating, or engaging in metacognitive activity. The framework used to code the cognitive demand of instructional activities is provided in Table 1.

Table 1 Criteria for coding the cognitive demand of mathematics tasks
Full size table

Two independent raters coded each task (64 in DU entries and 32 in AU entries); the overall agreement was acceptably high (80 and 70% respectively); any instances of disagreement were discussed, and a consensus rating was derived. In Table 2 we provide examples of tasks classified as high-demand or low-demand, along with a brief rationale for our decision in each case.Footnote 4

Table 2 Examples of tasks coded as high-demand and low-demand
Full size table

Pedagogical Features of NBPTS Portfolio Submissions

We focused on four pedagogical features identified in the mathematics education reform literature as being innovative and having the potential to cultivate the development of students’ mathematical understanding: tasks that involved multi-person collaboration and communication, considered applications in contexts other than mathematics itself, employed technology, or used physical (hands-on) materials. Because a teacher’s explanation of instructional context was generally not task-specific for each of the two tasks in a DU entry, we treated the entire DU entry, rather than each activity, as the unit of analysis for the coding of pedagogical features. Thus, 64 items (rather than 96) were coded in this analysis—32 AU entries and 32 DU entries. Agreement was nearly unanimous in the classification. Table 3 displays the judgment criteria we used in the coding and a portfolio entry excerpt providing evidence of the presence of that feature.

Table 3 Criteria and sample excerpts used in coding pedagogical features
Full size table

Relating Mathematical and Pedagogical Features of the Portfolio Entries

We examined the extent to which teachers in our sample used the pedagogical strategies in association with high-demand and low-demand tasks. For the 32 AU and 32 DU portfolio entries, we created 2-by-2 contingency tables, crossing cognitive demand (high or low) with pedagogical feature (present or absent). For each pedagogical feature, each contingency table displayed the number of teachers in our sample who submitted entries that were coded with the corresponding pair of characteristics. For the DU entries, we collapsed the cognitive demand coding for the two submitted activities, and we considered an entry to be high-demand if it contained at least one task that was coded as high-demand. We analyzed the data in these tables using chi-squared tests.

Relating NBPTS Certification Status to Mathematical and Pedagogical Features

To ascertain the interaction between the NBPTS view of high quality teaching and each of the other two views considered in this chapter—the cognitive demand view and innovative practice view—we examined the extent to which teachers in our sample who were awarded (or not awarded) NBPTS certification included (or did not include) high-demand tasks and reflected the presence (or absence) of each pedagogical feature.

As with the other similar analyses, we created 2-by-2 contingency tables, crossing NBPTS certification status (awarded or not awarded) with cognitive demand (high or low) and also with pedagogical feature (present or absent). Each contingency table displayed the number of teachers in our sample who submitted entries that were coded with the corresponding pair of characteristics. We analyzed the data in these tables using Chi-square tests.

Findings

Without knowledge of the NBPTS certification status of applicants, trained coders examined portfolio entries with respect to cognitive demand of the mathematics tasks and presence of innovative pedagogical features. After the portfolio entries were completely coded, we used these judgments to contrast the portfolio entries submitted by the 13 applicants who were awarded NBPTS certification with those submitted by the 19 applicants who were not awarded certification. We report our findings in this section: first with respect to the cognitive demand of the mathematical tasks, next with respect to innovative pedagogical features, and finally with respect to the interaction between cognitive demand and innovative pedagogy in the two sets of portfolio entries.

NBPTS Status and Cognitive Demand

Overall, 17 teachers (slightly more than half of the sample) submitted at least one high-demand task – 6 teachers submitted exactly one such task, 8 submitted exactly two such tasks, and 3 teachers submitted all three tasks that were judged to be cognitively demanding. Thus, 15 teachers submitted only low-demand tasks.

Figure 3 shows the percent of NBPTS certified (and non-certified) teachers who submitted (or did not submit) at least one high-demand activity. These data suggest a strong association between NBPTS certification and the submission of cognitively demanding tasks. In particular, four of every five teachers who submitted exclusively low-demand tasks in these two portfolio entries were not awarded NBPTS certification. Similarly, only one in four teachers who obtained NBPTS certification submitted exclusively low-demand tasks in the two portfolio entries we examined; that is, three-fourths of the teachers who obtained NBPTS certification submitted at least one high-demand task. A chi-square analysis indicated a statistically significant association (χ2 (32, 1) = 4.98; p < 0.05) between a teacher’s NBPTS certification status and the inclusion of at least one cognitively demanding task in his or her portfolio.

Fig. 3
figure 4_3_162454_1_En

Percent of teachers submitting (or not) high demand tasks by NBPTS certification status

Full size image

NBPTS Status and Innovative Pedagogy

Across the portfolio entries we observed much more frequent use of innovative pedagogical approaches than we found cognitively demanding tasks. Overall, the percent of teachers submitting at least one portfolio entry exhibiting each of the innovative features ranged from 100% for the use of contexts outside mathematics to about 60% for the use of technology, with 84% using hands-on activity and 66% including a task that called for collaborative activity. Table 4 shows the distribution of NBPTS certified (and non-certified) teachers who submitted (or did not submit) at least one activity that contained each of the pedagogical features we considered in the portfolio entries we examined.

Table 4 Number of NBPTS certified and Non-certified teachers giving evidence of using pedagogical features in at least one portfolio entry
Full size table

Because innovative pedagogy was so prevalent in the portfolio entries, these data suggest no more than a weak association between NBPTS certification status and use of the pedagogical features we examined. In fact, certified and non-certified teachers used three of the four pedagogical practices—the use of hands-on activities, contexts outside mathematics, and collaboration—in roughly the same proportion. Only in the case of technology usage was there some difference, with teachers who were awarded NBPTS certification employing this pedagogical feature more frequently. About three of every four teachers who obtained NBPTS certification employed technology in at least one of the two portfolio entries; non-certified teachers were about as likely to submit as to not submit an entry that used technology. Nevertheless, even in the case of technology use, the chi-squared analyses we performed did not indicate that any of these relationships or trends was statistically significant.

Cognitive Demand and Innovative Pedagogy

We also examined the interaction between cognitive demand and innovative pedagogy. This is an analysis of the extent to which teachers appeared to use innovative pedagogy in support of, or at in close association with, cognitively demanding mathematics tasks. Table 5 shows the frequency of each innovative pedagogical feature in the portfolio entries of teachers who submitted (or did not submit) at least one cognitively demanding task.

Table 5 Number of teachers submitting activities with pedagogical feature by level of cognitive demand of the portfolio entries
Full size table

From the data displayed in Table 5, we can see that there appears to be no overall relationship between cognitive demand and pedagogical innovation. That is, pedagogical features were detected about as frequently in portfolios in which high-demand activities were included and in portfolios in which no high-demand activities were present. Although the teachers in our sample used innovative pedagogy, these results suggest that they were not using these teaching practices in any systematic way to support students’ engagement with cognitively demanding mathematics tasks.

NBPTS Status, Cognitive Demand and Innovative Pedagogy

Although the overall data do not indicate a relationship, the picture might change if we also included NBPTS certification status in the analysis. Table 6 shows the frequency of each innovative pedagogical feature in the portfolio entries of NBPTS certified (or non-certified) teachers who submitted (or did not submit) at least one cognitively demanding task.

Table 6 Number of teachers by NBPTS certification status submitting activities with pedagogical feature by level of cognitive demand of the portfolio entries
Full size table

Similar to Table 5, the display of data in Table 6 suggests that there is no clear relationship between cognitive demand and pedagogical innovation when NBPTS certification status is considered. For example, the three NBPTS certified teachers whose portfolio entries did not contain any cognitively demanding tasks submitted activities that used all of pedagogical features (with the exception of one teacher who omitted hands-on activities). The pattern of usage was less uniform for the NBPTS certified teachers whose portfolios contained cognitively demanding tasks, and also for the teachers who were not awarded NBPTS certification, but no clear pattern emerges from the data. Also, because the numbers are so small in the subgroups when all three dimensions are considered simultaneously, we were unable to detect any statistically significant trend for any individual pedagogical feature in relation to cognitive demand and NBPTS certification status simultaneously. We also used cluster analysis considering the total number of pedagogical features present in a portfolio entry in relation to the presence/absence of cognitive demand and the NBPTS certification status of the teacher who submitted the entry. This analysis did not detect statistically significant differences, but it did suggest that the teachers awarded NBPTS certification tended to be more consistent (i.e., had less variance) than their counterparts who did not receive NBPTS certification in the use of pedagogical features in association with cognitively demanding tasks.

Discussion

Our goal in this study was to probe empirically the extent to which samples of teaching practice associated with a view of highly accomplished mathematics teaching as defined by the National Board for Professional Teaching Standards also exhibited characteristic features associated with two alternative views of high quality mathematics teaching: (a) the effective use of cognitively demanding mathematics tasks and (b) the use of progressive pedagogical practices. Toward this end, we examined samples of classroom instruction—lesson artifacts and teachers’ commentaries on lessons—submitted by 32 applicants seeking NBPTS certification. The instructional samples were systematically coded with respect to evidence of cognitively demanding mathematical tasks and innovative pedagogy. Finally, we examined the coded data to detect interactions between and among the different views of high quality mathematics teaching.

Our analyses detected a fairly strong interaction between the NBPTS view of accomplished teaching and the view of effective mathematics instruction associated with cognitively demanding tasks. In particular, we found that the teachers who were awarded NBPTS certification were far more likely than their colleagues who were not awarded certification to include high-demand mathematics tasks in the portfolio submissions we examined. Although these two views appear to be related in samples of actual instructional practice we examined in this study, they are clearly not identical. Recall that the decision to award certification is made on the basis of a composite judgment involving ten independent performance indicators and that the judgment of these performances did not explicitly attend to the issue of cognitively demanding mathematics tasks. In fact, when we examined the scores assigned by NBPTS raters to the portfolio entries in our study in relation to our coding of those same entries, we found that the two rating approaches were judging different aspects of the submissions. For example, 17 DU portfolio entries contained two low-demand activities, yet 65% of these entries received “accomplished” scores (a score 3 or greater) from the NBPTS assessors. Thus, the presence of low-demand tasks did not reliably predict a low assessor score on a particular entry, even though they appear to be related more generally to a low total score for the entire NBPTS process. Thus, our findings suggest that these two views of high quality mathematics teaching are related in the practice of teaching, but the relationship is complex.

The picture that emerges from our data analyses regarding innovative pedagogy suggests a different story. The innovative pedagogical features we examined—applications in contexts other than mathematics, multi-person collaboration, technology, or physical (hands-on) materials—were heavily used by the teachers in our sample, regardless of either their NBPTS certification status or their use of cognitively demanding tasks. Although we found that teachers used innovative pedagogical strategies in their classrooms, they did not do so in a way that was closely linked to supporting students’ encounters with challenging tasks. Even in our highly select sample of teachers who applied for NBPTS certification—thereby indicating that they thought of themselves as potentially highly accomplished teachers—we found little evidence that innovative pedagogy was used to support students’ engagement with cognitively demanding tasks. Such findings are consistent with some other research studies (e.g., Cohen, 1990; Ferrini-Mundy & Schram, 1997), and many anecdotes, suggesting that teachers may implement reform pedagogy in a superficial manner that does not realize its potential.

These findings appear to suggest that there is essentially no connection between pedagogical innovation, as defined here, and either the NBPTS view of highly accomplished mathematics teaching or the use of cognitively demanding mathematics tasks in instruction. Yet, we did find an interesting interaction. The teachers in our sample who not only were awarded NBPTS certification but also submitted at least one cognitively demanding mathematics task appeared to be more consistent than were other teachers in our sample in the use of innovative pedagogy. Though we did not find statistically significant differences, the suggestion of a difference regarding consistency of usage is worth pursuing in follow-up studies with larger samples.

Our investigation of the portfolio entries was not intended to be a validation study of the NBPTS certification process, and a replication involving a larger sample would be needed to make strong claims. Nevertheless, some of our findings do offer some validation of that process. In particular, the lack of correspondence between the awarding of NBPTS certification and the use of pedagogical features can be taken as evidence that the portfolio evaluation process is not heavily influenced by possibly superficial implementation of pedagogical innovation. And the positive association of low-demand mathematics tasks with non-certified teachers and high-demand mathematics tasks with certified teachers suggests that there is some reason to think that the instructional practice of those teachers awarded NBPTS certification is in fact “highly accomplished” in one mathematically important way that is not an explicit part of the NBPTS certification process. Moreover, the finding that at least some of the innovative pedagogy was used in connection with high-demand tasks by NBPTS certified teachers and not by those who were not awarded certification provides yet another indicator that the NBPTS certification process is reasonably well aligned with some other views of high quality mathematics teaching.

Given research evidence indicating both that teachers in the middle grades find it difficult to enact cognitively demanding tasks in mathematics instruction (Stein et al., 1996) and that the consistent, effective use of cognitively demanding tasks in the mathematics classroom increases student achievement (Stein et al., 1996), our findings suggest that there may be something to learn from NBPTS certified teachers about how to utilize such tasks effectively in the mathematics classroom. According to our analysis of the data examined in this study—teacher-selected samples of practice chosen by individuals seeking special recognition—the teachers who were awarded NBPTS certification appeared to deploy cognitively demanding tasks more proficiently than did their counterparts who were not awarded NBPTS certification. One caveat worth noting, however, is that we used a generous criterion when coding for cognitive demand—if some part of an activity exhibited high demand characteristics, it was classified as highly demanding, even if other parts of the activity did not. If we had applied a more stringent criterion—such as requiring that more than one half of an activity was judged to be cognitively demanding—the number of portfolio entries containing high-demand tasks would have been considerably smaller. Nevertheless, even if we applied a more stringent criterion, some of the activities submitted by the teachers awarded NBPTS certification were quite demanding and would likely have been so judged. Thus, it is left to future research to determine how robust the relationship detected in this study would be if more samples of instructional practice were examined and if different criteria were applied. But our findings clearly suggest a strong interaction between these two different views of highly accomplished mathematics teaching.

In the interest of supporting other research inquiry, we wish to underscore two special aspects of the data analyzed in this study that we think merit attention from researchers seeking to understand high quality mathematics teaching. First, the lesson materials and artifacts analyzed in this study were selected by teachers and submitted for evaluation in a process intended to identify highly accomplished teaching. Thus, it is reasonable to assume that the samples represented lessons that the teachers considered to be their best practice. In large-scale observational studies of teaching and in surveys, it is common to request samples of or information about typical teaching practice. Some scholars (e.g., Silver, 2003) have suggested the potential value of also examining instruction that is atypical in some way to detect, for example, what teachers might be capable of doing or inclined to do when they try to exhibit their very best work. The NBPTS portfolio entries offer one example of what such atypical data might look like, and our analysis of these data offers one example of what might be learned.

Second, the data examined were of a hybrid form that combines some features of the data collected via direct observation and data collected via survey responses. Like direct observation, the portfolio entries displayed important details of classroom lessons; similar to survey data, the portfolio entries permitted access to the teacher’s perspective. Although the NBPTS portfolio data might appear to overly limited as a source of information about teaching practice because the records do not include direct observation of actual teaching, the data in the NBPTS portfolio submissions are in many ways quite similar to those that have been used and validated by other researchers to study classroom practice using alternatives to direct observation and survey methods, such as “scoop” sampling of instructional artifacts (e.g., lesson plans, student work) to characterize instructional activity (Borko, Stecher, & Kuffner, 2007) and using classroom assignments to judge instructional quality (Clare & Aschbacher, 2001; Matsumura, Garnier, Pascal, & Valdés, 2002). Researchers interested in alternatives to direct observation methods (which are invasive, labor intensive, expensive, and impractical on a large scale) and survey methods (which involve questions susceptible to multiple interpretations, have questionable validity, and provide little information about the details of instructional lessons) might be wise to consider data like those collected in the NBPTS portfolio process to open another window on classroom instructional practice.

At the outset we noted that different views of high quality mathematics teaching are typically treated in isolation from each other, emphasizing the distinctions between and among them rather than the ways in which they might interact with or complement each other. In this chapter we examined the interactions among three ways of characterizing high quality mathematics teaching, and we identified some patterns observed in the interactions detected in samples drawn form actual classroom instruction. We hope that our report will stimulate further research that probes characterizations of high quality mathematics teaching to generate additional insights.