An enduring challenge of research in mathematics education is understanding and explaining the development of preservice teachers (Ambrose 2004; Ball 1989; Eisenhart et al. 1993; Mewborn 2000; van Es 2011). In their recently adopted Standards for Preparing Teachers of Mathematics, the Association of Mathematics Teacher Educators (AMTE 2017) acknowledges the “many areas in which research is not yet sufficient to determine the specific knowledge, skills, and dispositions that will enable beginning teachers to be highly effective…” (p. xii). Thus, research on preservice teachers is important to advance our understanding of the foundational knowledge, skills, and dispositions needed, especially in an era when teacher education is under increased scrutiny by policymakers searching for evidence that connects teacher preparation to student learning outcomes. Toward this end, researchers have begun the effort to improve teacher education by examining methods and conditions that might develop the knowledge and skills preservice teachers need to effectively teach mathematics (Blömeke et al. 2014; Darling-Hammond et al. 2007; Henry et al. 2014).

We examined the development of preservice elementary teachers’ (PSETs) professional noticing, their attitudes toward mathematics, and their mathematical knowledge for teaching. PSETs are of particular interest to researchers and teacher educators because they are typically prepared as generalists rather than specialists; thus, their coursework covers the spectrum of subjects rather than specializing in one content area (Ball 1989; Ebby 2000). Preparing them to teach mathematics is, therefore, a challenge in terms of available time in a curriculum of multiple important, yet in some ways, competing interests. Additionally, PSETs do not necessarily choose to teach mathematics, but rather, it is something that is part of the typical elementary teaching role and, therefore, the decision to teach mathematics is generally beyond their control. Often, PSETs have struggled in their own mathematical knowledge and have less favorable attitudes toward mathematics (Beswick 2006; Quinn 1997; Wilkins 2008). It is critical for teacher educators to build upon PSETs’ entering experiences to strengthen their teaching skills and build a more positive attitude toward mathematics (Ambrose 2004; Jong and Hodges 2013). This claim is echoed in recent organizational reports from the Teacher Education Ministerial Advisory Group (2014) in Australia, and in the USA, the National Council of Teachers of Mathematics (NCTM 2014), and the aforementioned AMTE (2017).

We created an instructional module, titled “Noticing Numeracy Now!” (N3), which focused on the three components of professional noticing: attending, interpreting, and deciding, as defined by Jacobs et al. (2010). Research has shown promising results in this area with in-service mathematics teachers (Jacobs et al. 2010); however, there is limited work, but growing interest, in this area with PSETs (Amador et al. 2017; Beattie et al. 2017; Dick 2017; van den Kieboom et al. 2017; Sherin et al. 2011). Amador et al. performed a case study in which they followed one PSET through three career points: preservice field experiences, student teaching, and first year of teaching career. Their results indicate that building professional noticing into preservice field experiences and scaffolding through the other two career points positively influenced the PSET in terms of student mathematical and scientific thinking. These results support the need for professional noticing instruction in preservice teachers. van den Kieboom et al.’s (2017) research, focused on PSETs’ professional noticing in the context of equality and the equal sign through diagnostic interviews, resulted in PSETs reporting an increase in their skill of professional noticing of children’s mathematical thinking about equality as well as their own understanding of equality and the equal sign. Such works indicate that situating PSETs’ professional noticing of children’s mathematical thinking in approximations of practice (Grossman et al. 2009) interacting with children might capitalize on PSETs’ nurturing attitudes while developing a deeper understanding of how children think about mathematics and, as Amador et al. describe, provides a foundation for growth in professional noticing as PSETs enter their career.

Our module, discussed in the following sections, focused on early numeracy content (i.e., children’s understanding of number and arithmetic reasoning); thus, we also set out to measure PSETs’ growth in mathematical knowledge for teaching in number and operations. We compared the professional noticing skills, attitudes toward mathematics, and mathematical knowledge for teaching of PSETs who were in a mathematics teacher preparation course where the N3 module was completed to another group of PSETs, also in a mathematics teacher preparation course, but one in which the N3 module was not implemented. We specifically examined the following research question: To what extent can an intervention focused on professional noticing of children’s early numeracy skills in mathematics methods courses enhance PSETs’ (a) professional noticing skills, (b) attitudes toward mathematics, and (c) mathematical knowledge for teaching mathematics relative to a comparison group that did not participate in the intervention?

Literature review and theoretical framework

Professional noticing

Generally, teacher noticing is a construct for the characterization and examination of responsive teaching practices. More specifically, professional noticing (of children’s mathematical thinking) as defined by Jacobs et al. (2010) is “a set of three interrelated skills: attending to children’s mathematics strategies, interpreting children’s understandings, and deciding how to respond on the basis of children’s understandings” (p. 172). Attending involves concentrating one’s attention on the students’ actions and verbalizations within a mathematical moment. For example, details worthy of attention might include a student’s movement of manipulatives, finger counting, or voice level. Interpreting involves an analysis of the observed behaviors or verbalizations with the aim of making some determination regarding the mathematical understanding of a student. Deciding refers to the teacher’s leveraging a particular interpretation to plan and enact a sound instructional or diagnostic course of action. Again, the three skills are conceived as interrelated and, as such, the component skill of deciding is highly connected to teachers’ interpretations of children’s mathematical thinking which are based on observed actions and verbalizations. In essence, these three components merge to form an interconnected process resulting in responsive teaching practice.

Certainly, other researchers have described seemingly similar phenomena but in different terms. For example, Mason (2002, 2011) used the terms accounting of and accounting for to describe key noticing processes (which are roughly analogous to attending and interpreting). Moreover, such noticing is, at times, described in terms of two components—identifying “what is noteworthy” and “making connections between specific events and broader principles of teaching and learning” (van Es and Sherin 2002, pp. 573–574). Interestingly, such depictions do not comment on the decisions and actions which follow. Delving more deeply, there are emerging arguments regarding the fundamental nature of professional noticing with respect to purpose (Schack et al. 2017). Indeed, some researchers have positioned professional noticing as practice aimed at capturing as much detail as possible about a mathematical situation (Schack et al. 2013; Wells 2017), while others describe noticing as a filter to identify the most impactful moments for action (Stockero and Rupnow 2017). Ultimately, we found a similar perspective of professional noticing put forth by Jacobs et al. (2010), as we found the inclusion of a decision as the natural outcome of professional noticing, to be a compelling structure for the consideration of not only teachers’ attention and consideration of children’s activities but also the manner in which these considerations shape subsequent instructional experiences.

Returning to the three-component perspective of professional noticing, Jacobs et al. (2010) found that teachers do not necessarily develop professional noticing skills solely as a result of teaching experience. Veteran teachers (12–14 years of experience) that had engaged in varying degrees of professional development focused on children’s mathematical thinking (Lamb et al. 2009) did not automatically develop skills in all three components of professional noticing. Note, this professional development drew heavily from the Cognitively Guided Instruction (CGI) effort (Carpenter et al. 1999) and consisted of five full days of workshops per year. Teachers that did not participate in this or any other professional development demonstrated emerging expertise in attending and interpreting but were more aligned with prospective teachers in the deciding component. From these findings, Jacobs et al. (2010) conclude that professional noticing skills are likely strengthened through teaching experience; however they are strongest after participating in professional development focused on children’s mathematical thinking.

Regarding teacher preparation programs, Grossman et al. (2009) explained how such programs tend to focus on the “preactive” elements such as lesson and unit planning and neglect the “interactive” elements necessary such as decision-making in the moment of teaching. Thus, viewing teacher preparation through the lens of professional noticing suggests an area of potential growth with respect to fostering PSET skills in making robust, evidence-based instructional decisions. Another challenge for preservice and novice teachers is that they are often expected to observe the classroom teacher but do not have sufficient pedagogical acuity to interpret the teaching and learning they are observing (Star et al. 2011). Star and Strickland (2008) assert that professional noticing should be a significant and early aspect of the curriculum for teacher preparation.

Stages of Early Arithmetic Learning

Numeracy is a fundamental content strand in PSET preparation and exploration (Van de Walle et al. 2012). In some instances, numeracy may be described quite broadly and include facility with the scientific method (Central Advisory Council for Education 1959). Similarly, the Australian Curriculum, Assessment and Reporting Authority (ACARA) (2014) states that “students become numerate as they develop the knowledge and skills to use mathematics confidently across other learning areas at school and in their lives more broadly…It involves students recognising and understanding the role of mathematics in the world and having the dispositions and capacities to use mathematical knowledge and skills purposefully.” Such expansive perspectives are consistent with the Programme for International Student Assessment’s (PISA) description of the mathematical literacy key competency (OECD/PISA 2005). However, other perspectives of numeracy are grounded, more specifically, in the foundations of whole number and operations (Mulligan et al. 1999). For example, Wright et al. (2006) describe such focus on whole number and operations in the context of assessment (referred to as the Learning Framework in Number) as early numeracy. For this study, we adopt a more narrow perspective of numeracy to examine noticing in the context of number and arithmetic reasoning.

The Stages of Early Arithmetic Learning (SEAL) is a highly descriptive progression that illustrates children’s development of this specific perspective of early numeracy via increasingly more sophisticated strategies involving the coordination of counting and arithmetic reasoning (Steffe et al. 1983, 1988; Steffe 1992; Olive 2001). Certainly, other scholars have differently portrayed this aspect of mathematical development ranging from the hypothetical learning trajectories put forth by Clements and Sarama (2009) to the landscape perspective of Fosnot and Dolk (2001). Particularly noteworthy is the Early Numeracy Research Project (ENRP) (Clarke 2001; Clarke et al. 2006). This project resulted in multiple frameworks to describe children’s development in early numeracy including counting and arithmetic reasoning. While complementary to SEAL at certain points, (i.e., the portrayal of counting on as a distinct conceptual milestone), the ENRP frameworks present a competing vision for mathematical development.

Turning to SEAL, this progression was constructed with six stages to capture and describe children’s early numeracy thinking: emergent counting, perceptual counting, figurative counting, initial number sequence, intermediate number sequence, and facile number sequence (Steffe et al. 1983, 1988; Steffe 1992; Wright et al. 2006). Each stage describes a child’s mathematical thinking at a particular point in his/her arithmetic development via the observed strategies. For example, perceptual counting refers to a child’s persistent reliance upon concrete materials (e.g., perceptual unit items) to facilitate arithmetic thinking. Figurative counting refers to a child’s capacity to enact slightly more abstract arithmetic strategies via quantitative mental imagery (e.g., representations) (Thomas and Tabor 2012). Ultimately, SEAL was selected as the mathematical perspective in which this study is grounded for two reasons. Given the remarkable detail put forth by Steffe and his colleagues as well as subsequent connections to instructional practice (Wright et al. 2006), we found that SEAL provides fertile ground for PSET professional noticing, specifically all three components. Further, the sustained presence of the US Math Recovery program (Wright et al. 2006) (which itself is grounded in SEAL) within the educational institutions of our area suggested that a focus upon SEAL would provide PSETs with a more consistent picture of children’s mathematical constructions. Toward this end, SEAL provided a mathematical lens through which teachers might make interpretations and decisions from the mathematical activities they observed among children. For example, should a teacher observe a child pointing to imaginary objects in the air as a means to determine the numerosity of a concealed collection of items, that teacher might leverage the language of SEAL to interpret these actions as indicative of a figurative counting scheme or motor representations (Steffe 1992). Further, the teacher might decide to pose addition tasks with a larger first addend (e.g., 18) and a smaller second addend (e.g., 2) aimed at helping the child develop a numerical composite and the arithmetic strategy of counting on. Such a decision would be consistent with the progression of stages put forth in SEAL. In summary, SEAL provides a productive mathematical framework from which teachers may structure key professional noticing processes.

Pedagogies of practice

While we have situated the conception of professional noticing within a particular mathematical context (numeracy development), Grossman et al. (2009) provide guidance regarding the development of these ideas within a teacher preparation program. Referred to as pedagogies of practice, Grossman et al.’s (2009) framework identifies different components with respect to teaching the practice of teaching. Specifically, the authors describe representations of practice, decomposition of practice, and approximations of practice.

Representations of practice are aimed at making the act of teaching visible and explicit to novice practitioners and may include video excerpts of specific teaching moments or “artifacts of practice…[such as] case records of clients, lesson plans, student work and live observations of practitioners” (Grossman 2011, p. 2838).

Decomposition of practice is described as “breaking down complex practices into constituent parts for the purposes of teaching and learning” (Grossman 2011, pp. 2838–2839). As such, the interrelated skills of professional noticing may also be considered a decomposition of practice in the sense that attending, interpreting, and deciding are smaller constituent parts of a responsive teaching practice.

Approximations of practice focus on providing opportunities “to engage in practice that is related, but not identical, to the work of practicing professionals” (Grossman 2011, p. 2840). This often hinges on a simplification of practice such as role-playing experiences or individual student mathematics interviews. It is important to note that approximations of practice should provide the occasion for targeted feedback. Grossman (2011) writes “In the buzz and complexity of classroom life, it is virtually impossible to pause interactions to provide such specific feedback which makes these approximations unique learning opportunities for novices” (p. 2840). In summary, we find that these pedagogies of practice provide a useful framework for the application of professional noticing within a program for teacher development.

Mathematical knowledge for teaching

Turning to other related frameworks, mathematical knowledge for teaching (MKT) appears theoretically connected to professional noticing. MKT denotes the diverse forms of knowledge necessary for effective mathematics instruction. This framework for classifying these different knowledge types (see Fig. 1) provides a useful organization of both content and pedagogical content knowledge related to the teaching of mathematics.

Fig. 1
figure 1

MKT domains (Ball et al. 2008)

Full size image

In this model, each domain denotes a particular variant of knowledge essential to effective mathematics instruction. Some of the domains are somewhat self-explanatory (e.g., common content knowledge); however, the knowledge invoked by other domains is, perhaps, less obvious. Knowledge at the mathematical horizon, for example, refers to an “awareness of how mathematical topics are related over the span of mathematics included in the curriculum…It also includes the vision useful in seeing connections to much later mathematical ideas” (Ball et al. 2008, p. 403). Likewise, specialized content knowledge refers to the “mathematical knowledge and skill unique to teaching” and is “mathematical knowledge not typically needed for purposes other than teaching” (Ball et al. 2008, p. 400). One instance of specialized content knowledge could involve the examination and evaluation of children’s early arithmetic strategies not just in terms of their solutions but also in strategic variability across differing aspects of number (e.g., symbolic, verbal, quantitative) (Thomas et al. 2010).

Attitudes and beliefs

To further a more comprehensive examination of PSET development, we also elected to examine the attitudinal development in the context of learning to professionally notice children’s mathematical thinking. For this study, we rely on Philipp’s (2007) definition of attitudes as “manners of acting, feeling, or thinking that show one’s disposition or opinion…Attitudes, like emotions, may involve positive or negative feelings” (p. 259). Conversely, beliefs are defined as “psychologically held understandings, premises, or propositions about the world” (Philipp 2007, p. 259).

Research has documented a positive relationship between teachers’ beliefs about mathematics teaching and learning and student achievement (Baumert et al. 2010; Hill et al. 2005). Within the field of teacher preparation, there have been inquiries into PSETs’ attitudes toward mathematics in the context of both clinical and course experiences (Schack et al. 2013; Quinn 1997; Wilkins 2008). Moreover, several studies have demonstrated a positive relationship between prospective teachers’ attitudes toward mathematics and their knowledge of common mathematical content (Matthews and Seaman 2007; Quinn 1997; Young-Loveridge et al. 2012). This relationship makes some intuitive sense as those individuals with stronger knowledge of content favor disciplines in which they feel more comfortable and confident.

Referring back to the framework of MKT (Ball et al. 2008), we recognize the complexity of knowledge with respect to mathematics teaching. Indeed, stronger knowledge of content, in and of itself, does not necessarily lead to effective teaching (Hill and Ball 2004). Furthermore, Wilkins (2008) examined multiple attitudinal factors and found that teachers with robust knowledge of mathematical content do not necessarily agree with more reform-oriented teaching practices. Perhaps a more important finding from this study, though, was that beliefs about mathematics might significantly influence teaching practices. Those individuals who believed mathematics to be a system comprised primarily of “transferrable” procedures and rules likely eschewed more inquiry-oriented teaching practices. Conversely, individuals who held more positive attitudes toward mathematics aligned themselves more closely to inquiry-oriented practices (Wilkins 2008). Cross (2009) provides evidence of the relationship between teachers’ pedagogical practices and their beliefs about teaching and learning. From this, it is conceivable that facilitating the development of positive attitudes and beliefs among PSETs may result in their adoption and emphasis of more conceptually oriented instructional tactics.

Methodology

Research design

The purpose of this quasi-experimental study was to examine whether an intervention influenced various outcomes; however, a random sampling methods was not utilized (Creswell 2014). Quasi-experiments are one of the few ways to test the effectiveness of interventions (Cook and Campbell 1979). We realize there are limitations to extraneous factors that can be controlled within educational settings (Shadish et al. 2001), especially given our uncontrolled before and after study which has a weaker evaluative design (Grimshaw et al. 2000). Given the nature of our research question, we still thought a quasi-experimental design was appropriate to examine the influence of an intervention with a comparison between groups. There were no ethical concerns regarding those who did not participate in the intervention because all were in mathematics methods courses that were likely informed by different frameworks. It might have also been the case that those in the comparison groups were exposed to professional noticing, but the course did not include the particular intervening module which is the focus of this study.

The N3 instructional module, which served as the intervention, focused on developing PSETs’ professional noticing of children’s early numeracy skills and was implemented at five institutions of higher education in a south central state in the United States of America (USA). The comparison groups consisted of elementary (grade levels K-5) preservice teachers from a similar population as those who participated in the intervention (Creswell 2014). Three separate assessments measured professional noticing skills, attitudes toward mathematics, and mathematical knowledge for teaching in 11 implementation groups and two comparison groups. The assessments were administered pre- and post-participation in the PSETs’ elementary mathematics methods coursework. Wilcoxon signed ranks tests were conducted to determine if a statistically significant growth existed post intervention, and paired t tests were conducted to compare changes between groups.

Instructional module description

We intentionally chose early numeracy as the context through which to develop the PSETs’ professional noticing skills. Given our primary aim of investigating professional noticing and the well-documented mathematical fragility of preservice elementary teachers (Ball 1990; Quinn 1997; Goulding et al. 2002), we sought to position this study such that the mathematical topics themselves would not impede (or minimally impede) PSETs’ noticing practices. Video cases of teacher-student exchanges provided opportunities to attend explicitly to and discuss salient features of children’s mathematical thinking often unnoticed by novice teachers in real-time classrooms. Because the video clips represented children’s thinking along the progression of SEAL, interpreting the nuances of the children’s thinking was supported by hallmark examples of each stage of SEAL. Decision-making was also supported by the common progression of children’s thinking outlined by SEAL. Using video of children as a precursor to engaging in their own diagnostic interviews with children provided opportunities for PSETs to anticipate and plan more specifically for what might occur in real-time classroom events.

The N3 instructional module, originally consisting of five class sessions and a culminating experience requiring the PSETs to conduct and analyze a diagnostic interview, was piloted at five institutions in seven mathematics education or mathematics courses the semester prior to initiating data collection. The N3 instructional module was revised after the pilot semester from five to four class sessions as feedback from PSETs was used to further refine and simplify the module. The module was embedded in the courses taught by the researchers whose teaching load regularly includes such courses.

Following the pilot and revision, data collection occurred in 11 course sections at five institutions which will be the focus of this study. After revising the module, those 11 course sections contained 577 preservice teachers that participated in the module. Participants were removed from the data set if they (1) did not give consent to use their responses for research, (2) had any missing scores from the three assessments, (3) did not have a matched pair with a pre or post, or (4) answered with responses indicating they could not correctly view the video or survey (i.e., “video would not load for me”). Once those entries were removed, this resulted in complete data sets for 224 preservice teachers.

The first session drew upon PSETs’ incoming awareness of the relative sophistication of various mathematical strategies, introduced the PSETs to the purpose and benefits of professionally noticing student thinking, and began the development of attending (the first of the three components of professional noticing). Decomposing professional noticing into its three interrelated components allowed us, the instructors of the courses, to emphasize the meaning and skill of each component. As the module progressed, the components were gradually nested and their interrelatedness acknowledged. The PSETs viewed video segments of early numeracy diagnostic interviews and recorded to what they attended. They engaged in partner, small group, and large group discussion and categorized each observation as teacher (or interviewer) actions, student actions (verbal and physical), and the mathematics of the task, recognizing overlap in these categories.

Session two focused on refining the PSETs’ attending skills and the value and challenges of video interview-based assessments. Additional video interview-based assessments were shown, and after each video, the PSETs, working in small groups, attended and recorded these “attendings” on a Venn diagram of four areas: teacher action, student responses, the mathematics of the task, and environmental factors (i.e., noise level, materials available, interruptions). Session three focused on using evidence attended to in seven video segments illustrative of SEAL, and knowledge gained from a SEAL homework assignment, to inform interpretations of children’s early numeracy thinking. PSETs discussed the salient attending evidence and made interpretations of the students’ SEAL stages using partner talk, small group, and large group discussions. PSETs used the information from these sessions and their out-of-class readings to complete a table each SEAL stage, including benchmarks and examples from videos.

The final session focused on all three interrelated skills of professional noticing in the context of SEAL, with an emphasis on the deciding component and the importance of attending and interpreting as a foundation for decision-making. PSETs viewed four video segments and were asked to attend, interpret, and decide about the next diagnostic or instructional steps based on their attendings and interpretations. The final activity of the session was an instructional deciding task where PSETs matched instructional strategies for advancing a focus student at each of the SEAL stages.

The culminating experience of the N3 module was an assignment that required the PSETs to conduct at least one diagnostic interview with a child, video record it, and analyze the video using the professional noticing framework. The evaluation of the assignment addressed each PSET’s ability to attend, interpret, and decide in the context of the interview conducted. The interview assignment varied somewhat across the universities but at all sites would be defined as an approximation of practice (Grossman et al. 2009). The interview assignment immediately followed the N3 module at some universities, while at others it was assigned later in the semester. All post-assessment data were collected at the end of the semester, following the interview assignment.

Participants for implementation sites

The participants at the implementation sites were predominately female PSETs who participated in the N3 module at one of five public universities in a south central state in the USA. Participants from two of the five universities are primarily from central Appalachia, an historically impoverished region with a traditionally underrepresented population in STEM fields. Participants from two of the universities came from a mix of suburban and urban environments. Although participants’ gender was not recorded, women were the predominant gender in each course which is consistent with the persistent (and growing) gender gap among US elementary school teachers (Dilworth and Coleman 2014).

The teacher preparation programs at the five institutions are relatively similar 4-year programs (or eight semesters) that prepare teachers to teach in schools for children between the ages of five and ten, or grades kindergarten through fifth grade. Teachers for these grade levels are prepared as generalists, that is they will be required to teach multiple subjects, including reading, writing, mathematics, science, and social studies. The mathematics coursework required of PSETs is, therefore, limited. In most programs, PSETs are required to take one general education mathematics course (e.g., problem solving or college algebra), two or three mathematics for teachers courses with content focused on the mathematics they will teach, and one mathematics instructional methods course with field experiences in local schools.

The participants were enrolled in one of over 20 mathematics or mathematics methods course sections over the study duration of 2 years. The mathematics course sections were hybrid courses combining mathematics for teachers and instructional methods. The courses, mathematics or instructional methods, are taken by PSETs prior to their culminating clinical semester, generally in the sixth or seventh semester of the eight-semester program. Thus, the PSETs involved in all sections were similar and the difference of curriculum between the types of courses was minimal.

Participants for comparison sites

The study involved two comparison sections of similar courses where the instructors did not teach the N3 module as part of their course. As the effects of the noticing intervention were as yet unknown, the creation of such comparison groups was consistent with quasi-experimental research designs. The comparison participants were 31 individuals who completed the same pre-/post-assessments as implementation site participants at the beginning and end of their elementary mathematics methods course or a similar content course for elementary education majors. Although one section of participants in the comparison group was at one of the same universities as the implementation sites, they were enrolled in a different section of the same elementary mathematics methodology course and did not experience any aspects of the professional noticing module. This first section of comparison participants was at one of the urban institutions in the study and the second section of comparison participants was at one of the suburban institutions, thus providing the matched diversity of the implementation sites. Similarly, as with the implementation sites, the comparison sites were comprised predominately of women enrollees.

Variables

Professional noticing

We designed an instrument to assess the PSETs’ professional noticing skills. The video-based professional noticing assessment was administered twice, once early in the semester and once near the end of the semester at both the implementation and comparison sites. At the implementation sites, the N3 module implementation occurred during the semester, after the pre-assessment and before the post-assessment.

The instrument consisted of a brief (25 s) video clip in which an interviewer poses a comparison, difference unknown task (Carpenter et al. 1999). This clip, despite its brevity, is not only rich in details of the child’s thinking that can be easily attended to but also includes nuanced details that might be missed by novices thus allowing for a range of scores. Also, the brevity of the clip allowed the PSETs multiple viewings, minimizing time constraints. In the clip, the interviewer has placed seven counting bears visible to the child as well as 11 small seashells that are screened by the interviewer’s hand. The interviewer tells the child he has seven bears, but too many shells, 11 shells, covers the shells, and asks, “How many shells will be left over” (if each bear has a shell)? The child correctly solves the problem by counting the visible bears starting at one, continuing the count on his fingers, and when he reaches 11, stops counting and announces the resulting difference between the two sets by realizing he has four fingers up at this point. The child does not need to count the four fingers; he recognizes there are four without looking. Full transcript as well as a screenshot of the video can be found in Table 1 and Fig. 2 (table and figure previously published in Schack et al. 2013). The PSETs viewed the clip and responded to three prompts. The three prompts were drawn from the work of Jacobs et al. (2010) and each prompt focused on one of the three interrelated components, attending, interpreting, and deciding. The prompts were (1) Please describe in detail what this child did in response to this problem, (2) Please explain what you learned about this child’s understanding of mathematics, and (3) Pretend that you are the teacher of this child. What problems or questions might you pose next? Provide a rationale for your answer. PSETs were expected to attend to the mathematical actions of the child (counting from one, using his fingers, counting on) (question 1) and use those attendings to interpret the mathematical ability of the child in relation to the operational thinking presented (addition skills, counting on, one-to-one correspondence, etc.) (question 2). Many PSETs chose to relate this question to the SEAL stage of the child in the video. The third question presented a chance for PSETs to provide an appropriate instructional or diagnostic response to the child’s ability that would be suitable for a child in his SEAL stage.

Table 1 Transcript and video description of assessment video clip (Schack et al. 2013)
Full size table
Fig. 2
figure 2

Professional noticing assessment video screenshot (Schack et al. 2013)

Full size image

We examined samples of PSET data for each of the professional noticing components for emergent themes (Glaser and Strauss 1967). The emergent themes were assimilated with researcher-identified key features for each of the components. The assimilation of themes and key features resulted in benchmarks that defined several ranked response types – four ranks for the attending component (score of 1–4) and three ranks for interpreting and deciding (score of 1–3) (see Table 2 for examples of each rank). Attending resulted in four ranks because the emergent themes from PSET responses showed a clear rank above the salient key features as defined by the researchers. In other words, a number of PSETs attended to details that were beyond what was determined by the researchers to be necessary for the interpretation, but were nevertheless astute responses, which contained pertinent information.

Table 2 Example professional noticing responses
Full size table

Initially, the researchers worked in pairs to analyze and rank responses. All researchers discussed and refined the benchmarks used. After several iterations of scoring with small samples of data, the researchers developed a flowchart designed to strengthen inter-rater reliability. The benchmarks provided the foundation for yes/no style questions on the flowchart used to guide the raters’ scorings. The resulting inter-rater reliability averaged 83% for all components across six scorers (Schack et al. 2013).

Validity measures

Due to the brevity of the video in the professional noticing assessment, extra precaution was taken to ensure construct and content validity of the measure. The professional noticing assessment was administered to 20 mathematics intervention specialists to determine the construct validity of the assessment. The mathematics intervention specialists were classroom teachers extensively trained in professional noticing of children’s mathematics through a statewide professional development initiative; therefore, we expected this group to score higher than PSETs on the pre-assessment. On all three components, there was a statistically significant difference in the percentages of specialists and PSETs that scored at each rank (attending: chi-square = 39.298, df = 3, p < .001; interpreting: chi-square = 27.977, df = 2, p < .001; and deciding: chi-square = 28.774, df = 2, p < .001). These results indicated that the assessment does assess an aspect of the construct of professional noticing.

To examine the content validity of the professional noticing assessment instrument, multiple experts of professional noticing or early numeracy reviewed the session materials and assessment for their usefulness and relevance to their respective areas of expertise. The reviewers included two lead editors of an award winning book on teacher noticing (Sherin et al. 2011), a co-author of multiple articles and books about SEAL (e.g., Wright et al. 2006), and a member of the design team of the Kentucky Numeracy Project (Kentucky Numeracy Project n.d.). The assessment video was rated by all reviewers as highly relevant, the highest ranking, or relevant, the second highest ranking, to early numeracy or professional noticing. Comments by the reviewers confirmed our design hypotheses and indicated the strength of the assessment video was in its simplicity, yet mathematical richness, as one reviewer stated, provided for “good potential to measure the impact of the instructional module.” Furthermore, the reviewers indicated that the scoring flowcharts contributed to the operationalizing of the components of professional noticing and “maintained a clear set of indicators for judging responses.” Based on the feedback of the reviewers, the assessment demonstrated that it was capable of procedures measures with content validity for professional noticing and early numeracy.

Attitudes Toward Mathematics Inventory

To measure the PSETs’ change in attitudes toward mathematics, the Attitudes Toward Mathematics Inventory (ATMI) (Tapia and Marsh 2004, 2005) was administered as a pre- and post-assessment at approximately the same time as the professional noticing assessment to participants at both implementation and comparison sites. The ATMI consists of 40 items, and each item is rated on a 5-point Likert scale from strongly disagree to strongly agree. Eleven of the items are reverse scored. Factor analysis by Tapia and Marsh resulted in four factors: value, enjoyment, self-confidence, and motivation (Tapia and Marsh 2005).

Tapia and Marsh (2004) originally developed the ATMI for use with secondary students, but it has since been used in post-secondary settings (Schackow 2005; Tapia 2012). The instrument used in this study was a version of the ATMI adapted for preservice teachers by Schackow (2005). The internal consistency of the modified instrument for this population was demonstrated by the Cronbach alpha coefficient of .98 with Schackow’s (2005) sample. Scores on the ATMI are determined by summing the results of items within each factor. The number of items per factor varies resulting in differing score scales for the factors.

Mathematics knowledge for teaching

The Learning Mathematics for Teaching (LMT) assessment was used as a pre-assessment and post-assessment with both the implementation and comparison groups to measure the PSETs’ MKT (Hill and Ball 2004). The LMT has multiple versions covering various content bands and topics. The “Numbers and Operations” version for grades K-6 was used, as its focus most closely aligned with the goals of the study though the items addressed a wider range of number and operations concepts, reaching beyond early numeracy. An online version of the LMT was administered that used computer adaptive testing to score and determine the next question in the sequence; thus, the length of the assessment varied for each PSET. The results of the LMT are scored using item response theory (IRT) revealing the number of standard deviations the score lies from the mean; thus, a negative score represents a score below the mean of the population of all teachers who have completed the assessment.

Results and discussion

The universities involved as implementation and comparison sites represented relatively diverse populations; therefore, there was initial concern among the researchers that PSETs at the various universities could begin at different levels and potentially impact results. Thus, before examining results of our research question, a Kruskal-Wallis test was conducted to determine if the data contained a significant difference in the starting scores at each of the universities. It was determined that there was not a statistically significant difference in the starting scores in the attending and interpreting components while the deciding component did yield a significant difference (attending: χ 2 = 12.084, df = 6, p = .06; interpreting: χ 2 = 12.182, df = 6, p = .058; deciding: χ 2 = 23.520, df = 6, p = .01). However, the same test was conducted on the change scores for each university and it was deemed that the PSETs at each university were growing at the same rate (no significant differences in their change scores); thus, initial concerns for possible skewed results due to the diversity across institutions were waived (attending: χ 2 = 8.818, df = 6, p = .184; interpreting: χ 2 = 7.409, df = 6, p = .285; deciding: χ 2 = 6.411, df = 6, p = .379). Because of these findings, all further analyses were conducted on two sets of data, implementation sites and comparison sites, but not stratified by universities.

Professional noticing: fertile ground for growth in PSETs

The results of the professional noticing ratings of both implementation (n = 224) and comparison sites (n = 31) are illustrated in Table 3 and provide evidence for answering an aspect of our research question focused on shifts in PSET professional noticing at implementation and comparison sites. The ratings reveal a shift in the percentages in the lower scores from pre- to post-assessment in the implementation groups while the comparison group percentages do not change as vividly.

Table 3 Implementation and comparison site professional noticing scoring percentages on pre- and post-assessments
Full size table

Wilcoxon signed ranks tests were conducted to determine if a statistically significant growth existed from the pre- to post-assessments in each of the professional noticing components at the implementation sites. A statistically significant difference occurred from pre- to post-assessments in all three components of professional noticing (i.e., attending, interpreting, and deciding) at the implementation sites (attending: z =  − 4.165, p < .001; interpreting: z =  − 5.521, p < .001; deciding: z =  − 7.229, p < .001). The results of Wilcoxon signed ranks tests on the comparison site data showed there were no significant increases from pre- to post-assessments in attending and interpreting; however a significant increase was found in the comparison site deciding scores (attending: z =  − 0.389, p = .697; interpreting: z =  − 0.894, p = .371; deciding: z =  − 2.021, p = .043). These results indicate that PSETs in a course where the N3 instructional module was implemented significantly improved on all three components while those in courses where the N3 instructional module was not implemented significantly improved on the deciding component but not on attending or interpreting. This result raises questions as to whether there is something other than attending and interpreting contributing to growth in deciding or, possibly that those PSETs who were instructed in the three interrelated components of professional noticing had learned to verbalize their “attending” and “interpreting” realizations more effectively while for those not instructed in professional noticing, attending, and interpreting remained more hidden and non-verbalized skills.

Given the near universal acceptance of the importance of differentiating mathematical experiences to accommodate and further the conceptual understanding of individual children, the framework of professional noticing seems well-positioned for use in teacher preparation programs. We conjectured that PSETs could develop professional noticing skills (attending, interpreting, deciding) as a result of their participation in a video-based module, and the results from this study appear to support this hypothesis within the context of early numeracy content. These findings seem to challenge the assertion of Jacobs et al. (2010) that “professional development that is sustained over not only months but many years” is needed for the development of professional noticing, in particular the “deciding-how-to-respond expertise” (p. 193). Our findings raise the possibility that learning the three interrelated skills of professional noticing may occur over a shorter time period. Using a video-based module that incorporated Grossman et al.’s (2009) pedagogies of practice, in particular the approximations of practice, gave PSETs the opportunity to experience teacher-student instructional practices in a guided environment resulting in growth in PSETs’ professional noticing skills. Given the call for incorporation of noticing pedagogies into teacher preparation programs (Star and Strickland 2008), the finding that PSETs can develop these skills as part of their participation in such programs is necessary for the substantiation and continuance of the enactment of professional noticing experiences by teacher educators. In summary, PSETs are not necessarily limited by lack of classroom experience and are capable of developing the interrelated skills of professional noticing around early numeracy; thus, we suggest that experiences designed around such skills are viable and appropriate pedagogies for helping PSETs learn to teach more responsively, especially when they are able to focus on a particular content.

Attitudes and beliefs: positive increases but minimal differences

The second part of our research question focused on shifts in PSET attitudes and beliefs and comparisons of such shifts across implementation and comparison sites. Table 4 shows the pre- and post-assessment means and ranges for implementation and comparison sites for the ATMI. The data show increases in all factors from pre- to post-assessment at both implementation and comparison sites.

Table 4 Implementation and comparison site attitudes toward mathematics inventory means and ranges
Full size table

Paired sample t tests were conducted on the data at the implementation sites and the comparison sites to determine if significant growth occurred in the four factors of attitudes toward mathematics. The results are displayed in Table 5. The results reveal that with the exception of the value component at the implementation sites, there was a statistically significant difference in the pre- to post-assessment of the factors of attitudes toward mathematics at both the implementation and comparison sites. In our previous research with a smaller sample size, it was determined that removing the scores of the PSETs who scored the maximum possible score on the value component in the pre-assessment, thus leaving no room for growth for those PSETs, yielded significant growth from pre- to post-assessment on the value component (Fisher et al. 2014). Similarly, for this study, there were 33 PSETs with a maximum score on the pre-assessment for the value component. When those PSETs were removed from the dataset, there was significant growth revealed (t = − 2.741, p = .007) from pre- to post-assessment in the value component for the implementation sites.

Table 5 Paired samples t test on attitudes toward mathematics factors on pre- and post-assessments
Full size table

We compared the change scores for each of the four factors between the comparison and implementation sites to determine if statistically significant differences occurred in the change of each factor between the groups. The results of the two-sample t tests revealed that there were no statistically significant differences in the change scores between the comparison and implementation sites in three of the components: enjoyment (t = − 1.931, p = .062), self-confidence (t = 1.99, p = .055), or motivation (t = 1.698, p = .099); however, a significant difference did occur in the change scores between the two groups in the value component (t = 2.105, p = .036) which favored the comparison sites. This significance in the change scores of the value component between the comparison and implementation sites could be attributed to the difference in the sample size of the two sites; however, since the removal of those with a perfect score on the value pre-assessment did yield significance from pre- to post-assessment for the implementation sites, the removal of those scores could also effect this correlation between the change scores of the comparison and implementation sites. It should also be noted that only two PSETs from the comparison sites obtained a maximum score on the value pre-assessment. In general, mathematics methods courses tend to have positive influences on PSETs attitudes toward mathematics. Thus, we cannot attribute such changes to the module administered within the implementation sites.

Mathematics knowledge for teaching: grounds for future research

The third part of our research question inquired about potential shifts in PSETs’ MKT at implementation and comparison sites. The PSETs in this study, overall, did not show growth in their MKT scores from pre- to post-assessment. Table 6 reveals the pre- and post-assessment descriptive statistics at the comparison and implementation sites on the LMT assessment. The statistics indicate that the average LMT score at the implementation sites slightly dropped from pre- to post-assessment while the LMT scores increased at the comparison sites, but all averages remained negative, indicating they were below the average in the population of participants using this assessment.

Table 6 Pre- and post-assessment LMT descriptive statistics
Full size table

When comparing the pre- and post-assessment scores for MKT, changes were not statistically significant at either implementation or comparison sites when a paired t test was used (comparison: t = 1.640, df = 30, p = .111; implementation: t = .892, df = 224, p = .374).

Final remarks

Professional noticing, as defined by Jacobs et al. (2010), is a set of interrelated skills that provides an achievable goal for supporting PSETs to develop responsive teaching practices. Through the results and discussion, we have shown data to support that PSETs who participated in the N3 instructional module demonstrated growth in professional noticing of children’s mathematical thinking in the context of early numeracy. The N3 module integrates theory with practice by situating the learning of professional noticing within a theory of the SEAL early numeracy learning progression. Rather than studying the early numeracy progression then applying in a classroom setting, the PSETs analyzed the practice of others and of themselves to further their understanding of how children learn mathematics, specifically early numeracy. The success of the N3 module in terms of integrating theory and practice mirrors the goals of the AMTE Standards for Preparing Teachers of Mathematics:

An effective mathematics teacher preparation program ensures that practice-based experiences, including mathematics methods courses and equivalent learning experiences, provide candidates with experiences using tools and frameworks grounded in research to develop core pedagogical practices and pedagogical content knowledge for teaching mathematics (AMTE 2017, p. 35).

Through the activities in the N3 module and the culminating assignment of a diagnostic interview of an elementary student, the PSETs engaged in multiple clinical settings of observing, practicing, and analyzing that progressively moved PSETs toward more independent work as teachers. Given Jacobs et al.’s (2010) results, it was not surprising to find that PSETs who participated in the intervention showed growth in professional noticing skills within the context of early numeracy mathematics content. We believe such growth is a promising finding that shows potential for PSETs to learn to professionally notice within a course. The next goal would be to determine if such learning impacts their professional noticing of children’s thinking in other mathematical learning progressions and in situations beyond their preservice experiences, such as reported by Amador et al. (2017).

Teacher educators may leverage professional noticing-oriented activities in content/methods courses to impact the future design of such courses. Such activities, focused along common learning progressions, provide PSETs with a foundation for shifting their attention to the assets of children’s thinking informed by the learning progression, and, in turn, the learning progression can inform diagnostic and instructional decision-making. For example, professional noticing in the context of SEAL provides PSETs with the ability to attend to the nuances of children’s mathematics such as the need for perceptual materials to count, to interpret the children’s work along the continuum of a common children’s learning progression, and to make effective instructional decisions targeted to the needs of individual children aiming for an accessible challenge for that child along the learning progression. While PSETs reported increased awareness of children’s thinking in the context of early numeracy and their responses to the professional noticing assessment prompts indicated a more nuanced knowledge of early numeracy, this did not translate into positive changes in PSETs mathematical knowledge for teaching as measured by the LMT. We believe the LMT is designed to capture a wider range of mathematics content in number and operations and thus did not zero in on the specifics of early numeracy.

A challenge of professional noticing in the N3 module is its focus at the individual student level. The activities of this module allow for detailed analysis of individual children, but not whole class settings. Developing professional noticing skills through multiple individual exchanges, video or otherwise, may provide a foundational repertoire for PSETs to anticipate children’s varying work and potential trends and clues that might occur in larger group instructional settings. While individual interviews potentially provide the most robust conclusions about individual student’s understanding or learning level, PSETs ultimately need to prepare for managing whole classrooms and time. Broadening the scope of the professional noticing framework to whole class settings, it is plausible that the interrelated skills of attending, interpreting, and deciding learned in the context of one-to-one teacher-student interaction could be harnessed to refine teachers’ selections of models of students’ mathematical thinking in open-ended problem solving situations (e.g., number talks) or to more effectively structure and sequence classroom mathematical discussions (Stein et al. 2008).

We believe it is valuable to explore professional noticing at a whole class level to broaden the impact in teacher preparation programs and practice. Our study, along with others (Jacobs et al. 2010; Goldsmith and Seago 2011), focused on cases of individual children’s mathematical thinking to develop professional noticing skills with teachers. Based on our own experiences and the requests of our PSETs, it is important that classroom scenarios be taken into consideration to develop a clear vision of the role professional noticing plays in practice. Moreover, the development of professional noticing of children’s mathematical thinking along an increasingly complex and comprehensive progression for the PSETs, from individual student to whole class, is in keeping with recent recommendations by the AMTE (2017).

In summary, we are cautiously optimistic about the potential for professional noticing of children’s mathematical thinking to serve as a mechanism for increasing the capabilities of PSETs to negotiate the nuances and complexities of mathematics teaching and learning. Our results contribute to the growing field of professional noticing and, perhaps more importantly, to the literature on teacher preparation. Situating professional noticing in the context of a well-defined trajectory of children’s mathematical learning might support PSETs in furthering their abilities to be more precise in their mathematics interpretations. While the relationships between professional noticing and other important constructs and frameworks (i.e., MKT, attitudes toward mathematics) remain unclear, we contend that further exploration in this area will likely illuminate additional avenues for strengthening the mathematical preparation of preservice teachers.